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ABSTRACT 



In response to Congressional direction, a quantitative, analytical review (a "meta- 
analysis") was completed of interactive videodisc instruction applied in Defense training 
and in the related settings of industrial training and higher education. Over all instructional 
settings and applications, interactive videodisc instruction was found to improve 
achievement by about 0.50 standard deviations over less interactive, more conventional 
approaches to instruction. This improvement is roughly equivalent to increasing the 
achievement of students at the 50th percentile to that of students currently at the 69th 
percentile. An improvement of 0.38 standard deviations was observed across 24 studies 
in military training (roughly an increase from 50th to 65th percentile achievement). An 
improvement of 0.69 was observed across 14 studies in higher education (roughly an 
increase from 50th to 75th percentile achievement). Interactive videodisc instruction was 
more effective the more the interactive features of the medium were used. It was equally 
effective for knowledge and performance outcomes. It was less costly than more 
conventional instruction. Overall, interactive videodisc instruction demonstrated sufficient 
utility in terms of effectiveness, cost, and acceptance to recommend that it now be routinely 
considered and used in Defense training and education. 
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SUMMARY 



A. PURPOSE 

The 1989 Department of Defense Appropriations Bill directed the Department of 
Defense (DoD) to conduct a study on the use of interactive videodisc technology in training 
and education as it pertains to effectiveness, cost-effectiveness, time on task, retention, and 
overall applicability to current and future DoD training and education requirements. 
Accordingly, a quantitative, analytic review-a "meta-analysis"--of what has been learned 
about the effectiveness and cost of interactive videodisc instruction was completed and is 
reported here. 

B. METHOD 

Empirical evaluation studies comparing interactive videodisc instruction v"th 
conventional instruction were collected from three instructional settings: military training, 
industrial training, and higher education. Results from these studies were th-n calculated 
using effect size, which is widely used in meta-analytic reviews and which provided a 
common, quantitative measure of instructional effectiveness mat could be applied across all 
studies considered. Effect size is a standardized index, measured in standard deviations, of 
the difference-in this case the difference in instructional achievement-be tween two 
treatment groups. It is discussed further in the body of the paper. In this review, the larger 
the effect size, the more effective interactive videodisc instruction was found to be than 
conventional instruction. 

C. FINDINGS 

The findings of this review principally concern the instructional capabilities, or 
functionalities, made available by interactive videodisc systems, not the hardware itself. 

1. Effectiveness 

Interactive videodisc instruction was used successfully to teach, four 
comparisons of interactive videodisc instruction with a "placebo" treatment in which no 
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relevant instruction was given, resulted in an average increase in achievement of 1.39 
standard deviations, or, roughly, an increase in the achievement for 50th percentile students 
to about the 92nd percentile of achievement 

Overall, interactive videodisc instruction was more effective than 
conventional instruction. Forty-seven comparisons of interactive videodisc instruction 
with conventional approaches were identified for this review. Over all instructional settings 
(military training, industrial training, higher education), instructional approaches 
(simulation only, tutorial only, simulation and tutorial combined), and outcomes 
(knowledge, performance, retention), interactive videodisc instruction increased 
achievement an average tf 0.50 standard deviations over conventional instruction (an 
increase for 50th percentile students to about the 69th percentile of achievement). 

Interactive videodisc instruction was more effective than 
conventional instruction in military training. Twenty-four comparisons of 
interactive videodisc instruction with conventional approaches to military maintenance, 
operator, and command training were identified for this review. Over all instructional 
approaches and outcomes, they showed an average increase in achievement of 0.38 
standard deviations (an increase for 50th percentile students to about the 65th percentile of 
achievement). 

Interactive videodisc instruction was more effective than 
conventional instruction in higher education. Fourteen comparisons of interactive 
videodisc instruction with conventional approaches to instruction in higher education 
settings (colleges and universities) were identified for this review. Over all instructional 
approaches and outcomes, they showed an average increase in achievement calculated 
using pooled standard deviations of 0.69 standard deviations (an increase for 50th 
percentile students to about the 75th percentile of achievement). The lower average effect 
sizes found for military training than for higher education may be due to a focus in military 
training on reaching threshold level (s) of achievement with minimised costs and time so 
that students who reach achievement criteria are sent on to duty assignments rather than 
held in the instructional setting a$ they are in higher education. 

Interactive videodisc instruction was equally effective for both 
knowledge and performance outcomes. The average effect sizes calculated using 
pooled standard deviations for 27 knowledge outcomes (facts, concepts, and other 
information students acquired) and for 20 performance outcomes (procedures, skills, and 
other capabilities students could demonstrate) both averaged around 0.35 standard 
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deviations (suggesting an improvement of 50th percentile students to about the 64th 
percentile). 

The more the interactive features of interactive videodisc technology 
were used, the more effective the resulting instruction. One study examined 
this issue directly and found greater achievement with greater levels of interactivity. The 
issue can also be examined by comparing the effect sizes for Level n videodisc instruction 
(-.70, overall) with those for Level III videodisc instruction (.50, overall). 

Directed, tutorial approaches were more effective than stand-alone 
simulations in interactive videodisc instruction. Effect sizes averaged 0.70 for 
tutorial approaches, 0.40 for combined tutorial and simulation approaches, and 0.15 for 
simulations by themselves. 

Within -group variability was smaller in interactive videodisc 
instruction than in conventional instruction. In nearly every comparison of 
interactive videodisc instruction with conventional instruction, achievement in the 
interactive videodisc instruction group was less variable than in the conventional instruction 
group, suggesting there was more equitable distribution of achievement using interactive 
videodisc instruction. 

Interactive videodisc instruction was more effective than computer- 
based instruction without videodisc interaction. Both interactive videodisc 
instruction and computer-based instruction have been found to be more effective than 
conventional instruction. However, the average effect size of around 0.69 observed for 
interactive videodisc instruction used in colleges is considerably higher than both the 
average effect sizt 0.26 found in a review of computer based instruction used in colleges 
and the average effect size of 0.42 found in a review of computer-based instruction used in 
adult education. 

There was little in the reviewed studies to indicate how interactive 
videodisc instruction achieves its success. The studies examined in this review 
did litde to indicate which features of interactive videodisc instruction contribute to the 
observed increases in student achievement Additionally, there are many outcomes-such 
as speed of response, accuracy of response, attitude toward the subject matter, insight, 
transfer, and retention-that may alone or in some combination become the objective(s) of 
instruction. How different designs for interactive videodisc instruction contribute to 
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accomplishing these different outcomes was rarely addressed by the reviewed studies and 
remains a proper topic for future research. 

2. Cost 

Interactive videodisc instruction was less costly than conventional 
instruction. All 13 cost ratios (the r*ao of costs for interactive videodisc instruction over 
costs for conventional instruction) found for this review were less than 1.0, indicating 
lower costs in every measured instance for interactive videodisc instruction. The average 
across all 13 cost ratios was 0.36. The average amount of student time saved across the 
studies covered by this review was 31 percent suggesting another source of savings from 
use of interactive videodisc instruction. Reductions in total military manpower that can 
result from such time savings in training may be an additional, significant source of cost 
savings, but this possibility was not examined here. 

3 . Cost-Effectiveness 

Interactive videodisc instruction was more cost-effective than 
conventional instruction. Interactive videodisc instruction was found to be both less 
costly in all studies that reported costs and more effective overall than the conventional 
approaches with which it was compared. This finding should be viewed as suggestive 
rather than conclusive since no studies were found that used systematic models to provide 
empirical data on both cost inputs and effectiveness outputs. 

4 . Time on Task 

Interact! vt videodisc instruction may increase time on task. Only one 
study reported an observation relative to this point. This study reported a 45 percent 
increase in the time spent practicing a targeted task as a result of the introduction of 
interactive videodisc instruction. This result was viewed as promising, but not conclusive. 

5. Retention 

Interactive videodisc instruction seems unlikely to effect retention. 
Four studies addressed retention. Both negative and positive results moved closer to zero 
over the retention intervals considered in these studies. Post-training experiences typically 
influence retention of knowledge and performance in a powerful manner. Findings for 
interactive videodisc training corroborate other Defense experiences with new training 

S-4 



9 

ERIC 



14 



approaches that suggest their value lies primarily in improving the efficiency of instruction, 
not in assuring longer retention. 

D. CONCLUSION 

The 47 studies reviewed here indie ^ that interactive videodisc instruction is both 
more effective and less costly than conventional instruction. They suggest, therefore, that 
interactive videodisc instruction can have a significant positive impact on the productivity of 
resources allocated to military training and education and on the availability of people 
needed to operate and maintain military systems. Although more needs to be learned about 
how interactive videodisc instruction should be designed and employed, it should now be 
routinely considered and used in military training and education. 
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I. INTRODUCTION 



A. PURPOSE 

The 1989 Department of Defense Appropriations Bill directed the Department of 
Defense (DoD) to conduct a study on the use of interactive videodisc technology in training 
and education as it pertains to effectiveness, cost-effectiveness, time on task, retention, and 
overall applicability to current and future DoD training and education requirements. 

DoD has been a leader in the development of interactive videodisc technology and 
has completed many evaluative studies of its applications in instruction. For this reason, a 
quantitatively-oriented, analytic review»a "meta-analysis"-of what has been learned about 
the effectiveness and cost of interactive videodisc applications in military training, industrial 
training, and higher education was undertaken and completed in response to the 
Congressional direction. 

B. BACKGROUND 

1 . Military Training 

Military training in the United States is a substantial enterprise. Some notion of the 
extent and scope of this activity is provided each year by the publication of the Military 
Manpower Training Report. This report describes how the trained manpower needs 
projected by another annual report, the Manpower Requirements Report, are to be provided 
in the coming fiscal year. 

The Fiscal Year 1990 Military Manpower Training Report (MMTR, 1989) reports 
that on an average day in FY 90 about 208,000 active duty personnel and 47,000 National 
Guardsmen and Reservists will undergo some form of formal training. This training load 
increased by about 8 percent in the 1980-1990 period. The cost of this effort will exceed 
$18.3 billion in FY 90, and it will require the support of about 176,000 military and 
civilian personnel to provide instruction, administration, and student supervision. 

However, the MMTR understates the magnitude of the total military training 
enterprise. It only concerns instruction conducted in formal courses by organizations 
whose primary mission is education or training. It excludes job-site training, factory and 



unit training for new systems, organized team training for performance of specific military 
missions, and all field exercises. The magnitude of resources allocated to these latter 
activities is difficult to determine, but it is likely to exceed the MMTR dollar projections by 
a factor of two or three. 

Military training is a complex as well as a sizable undertaking. At least six factors 
contribute to this complexity: 

(1) The supply of people available for military service is decreasing. In the period 
1978-1990, the number of people reaching age 18 each year dropped from 
about 4,250,000 to about 3,150,000. At present the Services must recruit 
50 percent of all male citizens who are not in college, not disqualified for 
mental, moral, or physical reasons, and are age 18-21; by 1995, this 
proportion could grow to 77 percent (Binkin, 1986). It will be particularly 
difficult to recruit people for high-skill military occupations since educational 
and/or second language difficulties will become increasingly common in the 
population of militarily eligible people. 

(2) The number of military systems is increasing. At the end of World War I there 
were about 500 materiel systems fielded by the the U.S. military, at the end of 
World War II this number had increased to about 2,000, and currently there are 
about 4,000 systems either fielded or in planning. In the Army there are about 
0.78 large systems per person-there is one wheeled vehicle for every four 
people, one tracked vehicle for every 20 people, one radio for every six 
people, one generator for every 10 people, and so on. If no technological 
changes were made in the complexity of military systems, their quantity and 
variety would by themselves substantially increase the demands on military 
training to provide the people needed to operate and maintain them. 

(3) The technological complexity of military systems is increasing. In 1939, the 
volume of technical documentation required for the J-F Goose, "Catalina 
Flying Boat" was 525 pages of information. In 1962, the volume required by 
the A-6A Intruder was about 150,000 pages of information. In 1975, the 
volume required for the F-14 Tomcat was about 380,000 pages of information. 
In 1979, estimates for the B-l bomber were in the neighborhood of 1,000,000 
pages of information (Kline and Gunning, 1979). This trend will doubtless 
continue. Whether the complexity of military systems necessarily translates 
into increased job complexity is a topic of continuing debate. Nonetheless, the 
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demand for people prepared to hold jobs classified as technical or highly 
technical has increased. The Army projected the number of people required in 
jobs categorized as "Very Technical" and •Technical" to increase by about 6 
percent from 1984-1990, and the Navy projected an increase of about 14 
percent in "Highly Technical" and •Technical" categories over the same period 
(Binkin, 1986). 

(4) Costs to conduct training in the field have risen in absolute terms and in terms 
relative to other DoD expenses. Fuel and ammunition have been major 
contributors to increased military training costs. Fuel costs will never return to 
pre- 1970 levels, and ammunition costs continue to rise along with the 
sophistication of new military systems. Additionally, die ranges needed to 
exercise the long reach of the newest systems are scarce, expensive, 
environmentally controversial, and have increased training costs substantially. 

(5) Modem warfare doctrine calls for more widely dispersed and more autonomous 
units. Access to training devices, subject matter expertise, qualified 
instructors, and training equipment will be substantially reduced by the need to 
maintain unit dispersion. Military trainers must increase their capacities for 
delivering training to dispersed and isolated job-sites-they must deliver 
training to students rather than deliver students to training. 

(6) Reserve component training poses particularly difficult requirements. The role 
of the Reserve Forces in the total force clearly shows that the Reserves are no 
longer in reserve. They will be among the first forces committed to battle in 
any future war. However, the Reserve components have a limited amount of 
time to train, units are widely dispersed throughout the country, units are not 
fully equippri, and only a small full-time force of qualified supervisors and 
trainers exists in Reserve units. 

Military trainers are aware of these challenges, and they have found that 
conventional approaches will not meet the new demands for increased training effectiveness 
without raising costs. Accordingly, they have sought new instructional approaches for 
meeting these new T ^quirements. It is not surprising to find them turning to our most 
powerful new tec Mogies for these approaches. Interactive videodisc technology is 
prominent among the new instructional approaches being tried 
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2. Videodisc Technology 

Like computer-based instruction, interactive videodisc instruction requires students 
to participate actively in the training environment, it provides practice with high quality 
simulation of devices that cannot be made available for training, and it distributes and 
standardizes both the content and interactions of high quality training to widely dispersed 
sites. Unlike computer-based instruction, interactive videodisc instruction also provides 
rapid, random access to a large, inexpensively stored data base of video quality images and 
sequences. 

Interactive videodisc technology links videodiscs and videodisc players with 
computers. Generally, the videodisc serves as a storage medium for the curriculum data 
base, and the computer controls how and in what order the curriculum material is presented 
to students. In this sense, teractive videodisc instruction is a form of computer based 
instruction, but the power and functionalities added by videodisc technology give it an 
identity of its own. 

Videodisc technology uses metalized plastic discs to code information as tiny pits 
(about 0.5 microns in diameter) pressed into a transparent substrate with a reflective 
coating. During play of the disc, the pits modulate a laser signal that is decoded into audio 
and video signals. About 15 companies possess manufacturing methods and processes to 
master and replicate such discs from a master videotape submitted by a curriculum 
developer. The first copy of a disc may cost $2,000-$3,000, but additional c, pies may 
cost less than $20 depending on quantity. DoD has specified a standard format, 
LaserVision, for production of all the videodiscs it uses. 

Videodiscs are available in several sizes-the most common are 8 inches and 12 
inches in diameter. The 12 inch interactive videodiscs can store 30 minutes of full-color 
video, or 54,000 video frames, on each disc side along v/ith two tracks of audio. With 
audio compression, 150 hours of audio can be stored on each disc side. Videodiscs can 
store digital as well as analog data. Depending on size and format they can store 200-3000 
MB (megabytes) of information. An interactive videodisc system is an audiovisual 
instructional system using a "personal" microcomputer, videodisc player, monitor, and 
special interfaces for displaying graphics and controlling the player. Such a system will 
generally cost $4 f 50O-$8,000. The videodisc player and its interface typically adds about 
$2,500 to a personal computer system. 
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Levels of interactivity~or the ability to respond to the actions of users-have been 
identified for videodisc systems of different configurations. Supplying trainers with a 
videodisc system at one level or another does not guarantee that it will be used in a fashion 
commensurate with its potential. This potential may or may not be fully used depending on 
the imagination and capabilities of instructional developers- However, these levels do 
identify progressively higher ceilings for interactivity. 

The following definitions of videodisc levels of interactivity were developed by the 
Joint Service Ad Hoc Committee for Interactive Courseware Data Item Descriptions. This 
committee based its definitions on those developed by the University of Nebraska 
Videodisc Design and Production Group, which in turn were derived from common 
developer usage. They are the following: 

• Level 0-A videodisc system intended for linear play without interruption. 

• Level I--A videodisc system with still/freeze frame, picture stop, frame and 
chapter search, dual channel audio, but no programmable memory. All 
functions are intended to be initiated by manual inputs from the videodisc 
player's keypad. Picture stop and chapter stop are read from the videodisc. 

• Level H--A videodisc system with on-board, programmable memory. The 
videodisc player's memory is programmed by "digital dumps" from audio 
channel two of the videodisc or by manual entry from the videodisc player's 
keypad. Inputs are made from the keypad or from a device that emulates the 
keypad. 

• Level III--A videodisc system in which the videodisc player is interfaced to an 
external computer. The videodisc player acts as a computer peripheral with its 
functions under the computer's control. 

• Level IV--A videodisc system in which the videodisc player is interfaced to an 
external computer. The videodisc functions both as an optical storage device 
for digital information and as the source of analog picture and sound. The 
video frames on the videodisc store digital data intended to be read and 
processed by the computer. 

Since the difference between Level III and Level IV depends on where and how 
digital information is storcd-in Level III it is generally stored on a digital device, such as a 
magnetic disk, and in Level IV it is stored on videodisc-whether or not Level IV provides 
the potential for more interactivity than does Level in is a topic of continuing debate. 
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The five definitions are listed, here for the record. Only two levels (II and IE) were 
found among the evaluations identified for this review. The term "interactive videodisc 11 in 
contrast with "videodisc" alone implies the presence of a computer, and it is used here to 
refer to Level in presentations. 

3. Interactive Videodisc Instruction 

New optical technologies based on compact discs such as CD-ROM (compact disc- 
read only memory), CD-ROM/XA (CD-ROM Extended Audio), CD-I (compact disc- 
interactive), DVI (digital video-interactive), and others still emerging (e.g., UVC 
Corporation's approach to video compression), as well as new approaches for computer- 
generated imagery, will compete for the market currently held by videodisc systems. 
However, the instructional capabilities, or "functionalities," of interactive videodisc 
systems are the co f cern of this review. Regardless of the hardware system eventually 
chosen to deliver them, the functionalities made available by interactive videodisc 
technology are likely to be preserved and pursued by the training community for the 
foreseeable future. 

These functionalities include those see"* in computer-based instruction. Drill and 
practice, tutorials, simulation, and gaming may all be enhanced by the ability of videodiscs 
to store on one randomly accessible medium substantial amounts of information in many 
different formats such as video still images, motion video, analog audio (on separate rid 
different tracks), and digital information such as compressed audio, curriculum data bases, 
and computer programming. 

Much of the development of instructional functionalities now found in both 
computer based instruction and interactive videodisc technology has been supported by 
DoD (Power On!, 1988). Fletcher and Rockway (1986) reviewed the history of these 
contributions and identified five new functionalities developed by the Department of 
Defense for interactive videodisc instruction. These five are the following. 

• Surrogate travel. This capability simulates a path of travel selected by the 
user over terrain lepresentations stored on videodisc. Areas chosen to 
demonstrate this capability have included a small city, a harbor, a nuclear 
power plant, and an art gallery. Using surrogate travel, the user chooses the 
path, controls the speed of advance, and directs the angle of view while 
"traveling" through an area using simple controk-usually a joystick. When 
the user come to a choice point such as an intersection, he/she can turn right, 
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turn left, proceed ahead, or go back. The videodisc frames that the user sees 
originate as filmed views of what one would actually see in the area-possibly 
including what one would see at different times of day or at different seasons 
of the year. 

Interactive movies. One problem with training that requires demonstrations 
of skilled performance is that essential components of the demonstrations may 
be invisible to viewers. Interactive movies are intended to solve this problem 
by allowing the user interactive control over many aspects of viewing such as 
direction (front, side, above, below), speed (fast, slow, still-frame, reverse), 
detail (panning and zooming), abstraction (photographs, labeled line-drawing 
animations), plot (different actions at different choice points), and 
simultaneous action (gauge readings, actions outside of the current angle of 
view). Interactive movies are prepared by^storing many different views of 
skilled performance on videodisc and making them available in parallel under 
the viewer's control. j 

Microt ravel. This capability combines s surrogate travel with interactive 
movies »nd allows travel in places where people cannot go. One 
demonstration of this capability for Defense training provided microtravel 
through a jeep engine while it was running. 

Spatial data management. Electronic libraries in the form of spatial data 
management systems allow users to "fly-over" spatially organized data using 
joystick controls. Data elements are associated with familiar terrain (e.g., a 
university campus) so that anyone familiar with the terrain can locate data of 
interest-information on chemistry would be found by "flying" to the chemistry 
department, information on tuba concertos might be found by "flying" to the 
tuba section of the university symphony orchestra found in the music 
department, and so on. The data may be in almost any form, and the spatial 
data management systems developed by DoD provided the earliest 
implementations and demonstrations of this approach to hypermedia 
presentations. 

Low-cost poriable simulators. One of the first military skill training 
videodiscs, developed in the late 1970s, was an Army tank gunnery trainer. 
This interactive videodisc simulation uses motion video segments stored on 
videodisc to provide tactical environments and targets. The student selects 



1-7 

22 



ammunition, aims, and fires using computer generated imagery overlaid onto 
the video display to receive round trajectory and burst on target feedback. As 
cm be seen from the evaluations discussed in this report, many other 
simulations were subsequently developed for military training. 

In Defense as elsewhere, training is a major application for interactive v : leodisc 
technology. About half of the approximately 100,000 videodisc systems in non-consumer 
use during the mid-1980s in the United States were used for training (Miller, 1987). The 
effectiveness of these applications is of substantial and natural interest both inside and 
outside DoD. Many organizations want to know what their investment in interactive 
videodisc instruction has bought in terms of increased ; .j-uctional effectiveness. That is 
the principal topic of the review reported here. 
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II. APPROACH 



This review is limited to an instructional medium-interactive videodisc. Therefore 
one criterion for including studies in this review was the use of a videodisc hardware 
system. The unambiguity of this criterion clarifies what studies were candidates for 
inclusion in this review, it aids replication, and it helps readers judge for themselves the 
relevance of this work to their own needs. 

On the other hand, the approach is subject to the cautions raised by many 
commentators and best articulated by Clark's (1983) critique of research on the 
instructional effectiveness of media. These concerns may be summed up by the notion that 
hardware alone does not define an instructional approach-what is done with the hardware 
is what counts. This point of view seems unequivocal. The presence of a computer 
controlled videodisc system is no guarantee of effective instruction nor even that the unique 
features of the system will be used. 

Critics of media effectiveness research push the argument farther. Clark states that 
"The best current evidence is that media are mere vehicles that deliver instruction but do not 
influence student achievement any more than the truck that delivers our groceries causes 
changes in our nutrition" (page 445). This statement may go too far. Improvements in the 
technology of delivering food from centers of production to markets has had a tremendous 
impact on the nutrition of nations. The technologies by themselves do not guarantee this 
impact, but the functionalities they support and their applications do. The development of 
movable type did not by itself have an impact on learning, but the functionalities it 
supported and their application to instruction did--no books, no mass education. 

This argument may extend to any medium, interactive videodisc included. The 
functionalities made possible and affordable by interactive videodisc technology hold 
substantial promise for instruction. Experimental investigation of these functionalities is 
essential to their effective application, and work concerning interactive videodisc 
functionalities such as that being done by Hannafin and his associates (e.g., Hannafin, 
1985; Phillips, Hannafin, and Tripp, 1988) and others (e.g., Gay, 1986) --even when it 
employs videotape rather than videodisc (e.g., Levenson, Morrow, and Signer, 1985; 
Tiene, Evans, and Milheim, 1988)»is needed, commendable, and to be encouraged, as 
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dark s article suggests. This review, however, is more summative. It is intended to bring 
together what we have learned thus far about the effectiveness of interactive videodisc 
technology applied to instruction. 

The following criteria guided the choice of studies for this review. 

(1) The studies had to conform to standard criteria for care and control in the 
conduct of evaluation research. These criteria are generally accepted and 
discussed in research literature concerning methodology for the design and 
conduct of experiments in the social and behavioral sciences. 

(2) The studies had io involve a comparison of an instructional approach using 
interactive videodisc with conventional instruction. "Conventional instruction" 
was assumed to involve all of the non-or limited-interaction approaches now 
available to instructors including platform lecture, text, programmed text, on 
the job training, experience with actual equipment, and even video and 
videotape instruction. It excluded highly interactive approaches such as 
computer-based instruction or interactive videodisc instruction. 

This requirement meant that the choice of studies was limited to 
comparisons of experimental and control group performance. It is possible, 
and perhaps desirable, to evaluate an instructional approach without these 
comparisons. For instance, tracing a "trajectory" of student progress through a 
course as described and applied by Suppes, Fletcher, and Zanotti (1976) 
directly relates time in instruction to achievement and is more powerful than the 
experimental group/control group comparisons sought here. However, 
comparisons were needed to calculate effect size, and studies without them 
were excluded. Some studies that only examined the effectiveness of different 
approaches to the design of videodisc instruction were excluded. Those that 
compared different levels of interactivity were included in this review, but 
others of this son were not. 

(3) The studies had to involve computer-controlled-Level Hi-videodiscs. Four 
studies were found that involved Level II videodiscs. These are reported and 
discussed briefly, but their results are not aggregated with the descriptive 
statistics reported for Level HI studies. Some good experimental studies 
turned up (e.g., Levenson, Morrow, and Signer, 1985) that used videotape 
players to mimic the interactive capabilities of instruction using videodisc 
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players. However, they dM not use videodiscs, and they were not included 
among the studies reviewed here. 

(4) The studies had to report sufficient data to permit calculation of effect size. 
Generally, means and standard deviations were needed for all treatment 
groups. In the absence of these data it was occasionally possible to estimate 
effect size from other information provided-especially information obtained 
from significance tests. Glass, McGaw, and Smith (1981) describe a number 
of ways to calculate effect size when means and standard deviations are not 
provided directly. These methods were used when they could be. 

(5) The studies had to concern military training or use students of about the same 
age and type as military training students. In practice, this meant that the 
studies were limited to military training, higher education, and industrial 
training. Only one study using non-handicapped elementary school students 
(Hasselbring et al., 1988) and only two using non-handicapped secondary 
school students (Blatnik and Holden, 1988; Gale and Barksdale, 1989) turned 
up. 

(6) All work with handicapped students was excluded. Research on videodisc 
applications for handicapped students has a long and vigorous history. Work 
by Hofmeister (e.g., Hofmeister, Engelmann, and Carnine, 1986), 
Thorkildsen (e.g., 1985), Browning (e.g.. Browning et al., 1986), and their 
colleagues is notable. A review of this work could probably be supported by 
the state of the art, but it is beyond the scope of this investigation. 

This investigation benefited from other reviews of interactive videodisc instruction. 
These reviews include those by Bosco (1986), DeBloois, Maki, and Hall (1984), DeBloois 
(1988), an annotated bibliography compiled by Sarii et al. (1988), and a review completed 
by Capper (in press). On the basis of these reviews, it was evident that the state of our 
knowledge would support a quantitatively oriented, analytic review of the effectiveness of 
interactive videodisc instruction. 

A. TECHNICAL APPROACH 

The methodology used for analytic reviews has changed considerably in the last 15 
years. The "box score" approach which earlier characterized the methodology has been 
replaced by "meta-analysis." In the box-score approach, studies in which an experimental 
group exposed to the treatment under review are collected, the proportion of studies in 
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which the experimental group means exceed control group means by some statistically 
significant extent is calculated, and the treatment is reported as favorable or not depending 
on whether this proportion is large or small. Hedges and OUrin (1980) have shown that the 
box score approach has very low power-low ability to detect statistically significant 
differences between two groups-for the treatment effect sizes and sample sizes 
characteristic of social science research. They also showed that the power of the procedure 
decreases as the number of studies included in the review increases. 

Glass (1976) proposed an alternative approach. Since he was performing an 
analysis of analyses, he described his alternative as "meta-analytic." It differs from the 
box-score approach in three ways: (1) studies relevant to the issue at hand are collected 
using clearly defined procedures that can be replicated; (2) a quantitativ measure, "effect 
size/' is used to tabulate the outcomes of all the collected studies including those with 
results that are not statistically significant; and (3) statistical procedures are used to 
synthesize the quantitative measures and describe the findings of the analysis. Glass's 
approach appears to be especially appropriate for synthesizing the results of instructional 
research, and it has been widely used for this purpose since its introduction. 

Meta-analysis is still developing as a technique and some matters concerning its use 
remain unsettled. The issues receiving the most attention at present concern the "file- 
drawer problem" and the calculation of effect size. The use of meta-analysis to perform 
research syntheses that include costs (syntheses involving cost-effectiveness) remains 
unaddressed and undetermined. 

The file-drawer problem may stem from the reluctance of professional journal 
editors to include studies whose results fail to reach statistical significance and the 
reluctance of researchers to report their null results-these studies, therefore, remain in file- 
dn< wcrs. The question, then, is how much effect would these inaccessible studies have on 
the results of our meta-analyses? After some analysis, the answer according to Rosenthal 
(1984)-among others-seems to be not much. Nonetheless, steps were taken in this 
review to locate studies left in file-drawers. 

Effect size is usually defined as the difference between the means of two groups 
divided by the standard deviation of the control group. Effect sizes calculated in this way 
estimate the difference between two group means measured in control group standard 
deviations. Glass et al. (1981) suggest that choice of the denominator is critical and that 
choices other than the control group standard deviation are defensible. However, they 
endorse the stanri*«i choice: using the control group standard deviation. 
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Alternatively, Hedges and OUrin (1985) show that, for every effect size, both the 
bias and variance of its estimate are smaller when the standard deviation is obtained by 
pooling the sample variance of two groups instead of using the control group standard 
deviation by itself. An effect size based on a pooled standard deviation estimates the 
difference between two group means measured in standard deviations estimated for the full 
population from which both experimental and control groups are drawn. Both approaches 
ait used in this review to estimate effect sizes and both are reported whenever sufficient 
data were available to calculate them. 

If the standard deviations for both control and experimental groups are about the 
same in magnitude, effect sizes calculated using control group standard deviations will 
differ little from those calculated using pooled standard deviations. If, on the other hand, 
effect sizes are larger when they are calculated using pooled standard deviations than they 
are when they are calculated using control group standard deviations, there must 
necessarily be smaller standard deviations and less variability across the measure of interest 
in the experimental group than in the control group. This is the case found here. The effect 
sizes calculated in this review using pooled standard deviations were generally larger than 
those calculated using control group standard deviations. 

Most commentators suggest that effect sizes can be treated as descriptive statistics 
and entered into standard tests for statistical significance. Hedges and OUrin (1985) have 
shown that the error variance around estimates of effect size is inversely proportional to the 
sample size of the studies from which the effect sizes are drawn. If the effect sizes in any 
review are cuawn from studies employing widely different sample sizes, then the 
heterogeneity of variance among effect sizes prohibits their use in conventional t-tests, 
analyses of » ariance, etc. In this review the sample sizes vary by a factor of 20; hence the 
effect sizes reported here are treated only with descriptive statistics. Standard deviations 
and sample sizes for effect sizes are reported so that others may examine the results further 
using such inferential nx r.z as they wish. 

B. DATA COLLECTION 

Defense Technical Information Center, Educational Resources Information Center, 
and Psychological Abstracts data bases were searched using all combinations of the 
following: 

[Computer <or> videodisc] 
<and> 
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[Assisted <or> Aided <or> Mediated <or> Managed <or> Based <or> <Empty>] 

<and> 

[Education <or> Learning <or> Training] 

Additionally, the following terms were used by themselves: 

• Interactive Videodisc 

• Interactive Video 

• Interactive Courseware. 

This search turned up more than 2,400 candidate studies. Most of these studies 
concerned computer-based instruction alone and not interactive videodisc instruction. This 
search was not without value, since it identified about two-thirds of the studies eventually 
used in this review. On the other hand, there were several studies that wot known to exist 
but that did not turn up in this search. Some of these studies are unpublished and not 
catalogued anywhere. O Jiers are published but not catalogued using any of the above 
keywords. Still others exist only as laboratory reports. Whatever the case, it was 
necessary to decide whether or not to include studies that did not turn up in the formal 
search process. A decision was made to include them. 

To locate uncatalogued, unpublished evaluation studies, we 1 contacted cognizant 
individuals in the training commands of all three Services (US Army Training and Doctrine 
Command, US Navy Chief of Education and Training, and US Air Force Air Training 
Command), in the personnel research and development commands of all three Services 
(US Army Research Institute, US Navy Personnel Research and Development Center, and 
US Air Force Human Resources Laboratory, among others), and in specific Service 
schools where we knew interactive videodisc materials had been used. 

We also contacted industrial training organizations to see if they had evaluation data 
and if they were willing to release it. We pursued this by publishing a notice in the 
Videodisc Monitor, by calling every developer listed in the March 1988 InfoWindow 
Courseware Catalog; and by calling every developer listed in the 1988-1989 Videodisc 
Compendium for Education and Training. With these actions we hoped to unearth 
"fugitive" documents that had reported studies performed for business but not formally 
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published. Few such documents emerged. We were gratified by the willingness of 
researchers to root through their personal files and dig out references that were already in 
the published literature. Some of these references had not been identified by our search of 
the data bases, and we were glad to know about them 

Finally, we telephoned a number of researchers directly to obtain original data to 
permit calculation of effect sires when the necessary data were missing from their report 
documentation. As usual we encountered helpful, facilitating people who were willing to 
dig through their personal files and send us whatever they felt would be relevant. 
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III. FINDINGS AND DISCUSSION 



A. TABULATION OF STUDIES 

The effectiveness of interactive videodisc applications in training and education can 
be assessed by examining the effect *izes reported in Appendix A. That appendix is 
intended to invite exploration of data collected for this review and (reexamination of the 
findings presented in this paper. 

Outside of what could be determined by descriptions of the instructional approach 
and what could be determined from the evaluation results, direct assessment of interactive 
videodisc content or quality was beyond the scope and resources of this investigation. 
Aspects such as the quality of graphics, clarity of instructional text, and verisimilitude of 
simulations were not assessed The state of the art in interactive videodisc instruction could 
most probably support such an analysis, and it would be an appropriate next step, but it 
was not attempted here. 

There are 28 studies summarized in Appendix A— some with more than one 
reference. The Appendix is organized so that the 13 military training studies are listed first, 
the 4 industrial training studies are listed next, and the 1 1 higher education studies are listed 
last Within these groups, the studies are listed chronologically. Two studies were 
performed at the US Air Force Academy (Crotty, 1984; Verano, 1987). Although these 
studies used military subjects (cadets), their setting seemed more characteristic of higher 
education than military training, and they are included with the higher education studies. 

Although 9 of the 17 studies in military and industrial training settings concerned 
electronic maintenance, instructional content concerned a variety of subject matter, 
including "softer" topics such as military operations in urban terrain (King and Reeves, 
1985), interpersonal skills (Schroeder et al., 1986), and handling of hazardous materials 
(Bosco and Wagner, 1988). None of the 1 1 studies in higher education settings concerned 
electronics maintenance. Their instructional content ranged from equipment operation 
(Fowler, 1980), CPR instruction (Lyness, 1987), and medical education (Allan, 1989) 
through biology (Bunderson, ct al., 1984, Davis 1985), physics (Stevens, 1984), and 
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chemistry (Smith et aL, 1986; Jones, 1987; Jones 1988), to foreign language instruction 
(Crotty, 1984; Verano, 1987). 

The main distinction in instructional approaches was among those applications that 
used the videodisc as a tutorial, programmed textbook with visuals-the intrinsic 
programming approach advocated by Crowder (1959)-, those that used the videodisc to 
permit free exploration of simulated situations or equipment, and those that ~*ed both by 
combining tutorials with simulations. Judgments concerning instructional approach were 
based on documentation of the instruction and on discussions with developers or others 
who had direct experience with it 

In Table A-l (Appendix A), the "N" column lists the number of subjects in the 
experimental (Exp) and the control (Ctrl) groups. 

The Comparison column tells what the experimental and control treatments were. 
Generally the experimental group involved interactive videodisc (IVD) presentations, and 
the control group did not. In some cases, different interactive videodisc presentations were 
compared with each other or computer assisted instruction (CAI) was compared with the 
control treatment. Findings from these comparisons are also included in Appendix A. One 
paradigm that is listed frequently in the Comparison column involves instruction using 
videodisc-based simulations of equipment compared with instruction using hands-on 
experience with the actual equipment (AE) that was the object of the instruction. 

As shown in the Outcome column, four categories of effectiveness data for 
interactive videodisc instruction were identified. These categories were: knowledge 
outcomes, which assessed students' knowledge of facts and/or concepts presented by the 
instruction; performance outcomes, which assessed students* skill in performing some 
criterion tasks or procedures; retention, which reported knowledge or performance 
measures taken some time after instruction ceased; and time to complete the instruction. It 
was assumed that time to complete a performance test was properly considered as a 
performance outcome measure, but that time to complete the instruction itself should be 
considered separately. 

Effect size (ES) was calculated as the difference in the group means divided both by 
the pooled standard deviation (Comb SD) and, where possible, by the control group 
standard deviation (Ctrl SD). In some cases, effect size was estimated from other statistics 
using procedures suggested by Glass et al. (1981). 

Some studies involved simultaneous comparison of more than two treatment groups 
and began with an analysis of variance (ANOVA) before comparing the treatment groups in 
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a pairwise manner. The statistical significance of the analysis of variance comparison is 
reported in the table. All effect sizes reported were calculated from pairs of treatments. 
Each effect size was calculated so that it was positive if the difference favored the videodisc 
treatment and negative if it favored the control treatment. More specifically, and to cover 
cases such as those in which different videodisc treatments were compared with each other, 
effect size was always calculated so that it was positive if it favored the instructional 
treatment judged to be the most interactive of the pair. The treatment judged to be most 
interactive is always the first of the two treatments listed in the corresponding Comparison 
column. 

The abbreviation "ns" is used in the Effect Size column to indicate that the observed 
difference in treatments was not statistically significant (p £ .05). Otherwise, the 
significance level reported in the study is listed. 

Superscripts are used in the "ES Comb SD" (Effect Sizes Based on Combined or 
Pooled Standard Deviations) column to indicate the categories into which the effect size 
was coded. Effect sizes based on knowledge outcomes are signified by a superscript "K", 
and those based on performance outcomes are signified by a superscript "P". Effect sizes 
based on time to complete the instruction are signified with a superscript "D" (duration). 
Only 8 studies in Appendix A reported time to complete instruction, so knowledge and 
performance outcomes were combined and coded together for these duration outcomes. 
Effect sizes based on retention are signified with a superscript "R". Only four studies in 
Appendix A measured retention, so knowledge and performance outcomes were also 
combined and coded together for retention outcomes. 

Superscripts are also used in "ES Comb SD" column to indicate how the effect 
sizes were coded with respect to instructional approach. Effect sizes judged to be based on 
tutorial approaches without simulation are signified by a superscript "T", those judged to be 
based on tutorial approaches combined with simulation are signified by a superscript "C", 
and those judged to be based on simulation alone are signified by a superscript "S". 

Four studies reported "Baseline" compHsons of interactive videodisc instruction 
with control conditions in which no instruction or irrelevant placebo instruction was 
provided. Generally, these studies tell r.s if interactive videodisc instruction teaches 
anything. Effect sizes concerning these baseline comparisons are signified with a 
superscript "B" in the appendix. 

Some studies used interactive videodisc instruction in both comparison groups but 
compared high or more extensive interactivity with low or less extensive interactivity. 
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However, several investigations listed in Appendix A measured more than one instructional 
outcome for the same experimental condition and for the same experimental subjects. 
These outcomes are non-independent, correlated estimators of true effect size and should 
not be treated as if they were independent. Hedges and Olkin (1985) present methods of 
estimating a common effect size from a vector of correlated estimators. Unfortunately, the 
correlations needed to apply their methods are rarely reported. Further, as Hedges and 
OUrin point out, the gain in efficiency resulting from using their multivariate methods 
seldom justifies the required effort An alternative approach is simply to average non- 
independent effect sizes across a single experiment, and that is the approach used here. 

In calculating these averages, it was necessary to decide what were separate 
experimental comparisons, independent and worthy of separate entry, and what were not. 
When more than one result was reported and more than one effect size was calculated from 
the same investigation, the following assumptions guided these decisions: 

(1) If the effect sizes were based on comparisons using different students/subjects, 
it was assumed they constituted different experiments and the effect sizes were 
not averaged. 

(2) If the effect sizes were based on comparisons involving different (more than 
one) experimental groups with the same control group, it was assumed they 
were different measures of different experimental conditions and the effect 
dzes were not averaged 

(3) f the effect sizes were based on comparisons involving the same experimental 
eatment (one experimental group) but different (more than one) control 
roups, it was assumed they were different measures of the same experimental 

< editions and the effect sizes were averaged. 

(4) Afferent experiments using the same interactive videodisc instruction materials 
vere not averaged. 

In A jpendix A, a superscript "X" in the "ES Comb SD" column signifies an 
average effr ;t size calculated for two or more measures reported for the investigation- 
"KX" sigmiies an average calculated for knowledge outcomes, "PX" signifies an average 
calculated for performance outcomes. "AX" signifies an overall average calculated for 
knowledge, performance, retention, and/or duration (time to complete the instruction) 
outcomes. 
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B. EFFECTIVENESS 

1. Principal Outcomes 

Effect sizes fcr the four principal measures of merit-knowledge outcomes, 
performance outcomes* retention of knowledge or performance, and time to complete the 
instruction-were averaged across all 28 studies and axe shown in Table 1. The effect sizes 
reported here and elsewhere are organized into three columns: 

• Column one shows the average effect size for all available studies for the row 
category. The effect sizes entered into this average are all based on effect sizes 
calculated from pooled standard deviations. 

• Column two also shows average effect sizes calculated from pooled standard 
deviations. However, they are based only on those studies that presented 
sufficient data to allow both pooled and control group standard deviations to be 
used in calculating effect size. 

• Column three shows average effect sizes calculated using control group 
standard deviations only. 

All the effect size averages in Table 1 that were calculated using pooled standard 
deviations are larger than those calculated using control group standard deviations. This is 
a stable finding in this review. It indicates that control group standard deviations were 
generally larger tl. ^n experimental group standard deviations and that interactive videodisc 
instruction raised average achievement more equitably across all students than did 
conventional instruction. The increases in achievement under interactive videodisc 
instruction were not due to a few students who were permitted to surge ahead in 
achievement (and increase the variance). This result is especially important for training 
where a principal concern is to raise as many students as possible across a threshold-or 
thresholds-of achievement rather than simply to maximize average achievement, which is 
accomplished most efficiently by concentrating instructional resources on the best students. 

The bottom section of Table 1 shows overall average effect sizes aggregated across 
all studies and all four measures of merit. The number of measures that contributed to the 
overall effect sizes reported in the table is less than the sum of measures contributing to the 
other effect sizes since some within-study measures of merit for a single experimental 
condition were averaged together. These within-study averages, which were used to 
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calculate the overall average effect sizes reported in Table 1 , are shown in Appendix A and 
signified with an H X" as discussed above. 

Table 1. Average Effect Slzee for Knowledge Outeomee, Performance 
Outcomes, Retention, and Instruction Completion Time 



ES Based on 
BadadSD 



ES Based on 
Pooled SD 
When CM SO 



ES Based 

nnftriSD 



Knowledge 
QUtfiOJDfia 

Mean 

SD 

N 



0.361 
(0.493) 
(27) 



0.375 
(0.506) 
(23) 



0.361 
(0.533) 
(23) 



Etftonnancfl 

Oleomas 



Mean 0.349 0.398 0.334 

SD (0556) (0.539) (0.447) 

N (20) (15) (15) 



Retention 



Mean 0.547 0.750 0.650 

SD (0511) (0.523) (0.339) 

N (3) (2) (2) 



Time to Complete 
Instruction 



Mean 1.178 1.267 1.185 

SD (0582) (0.474) (0.570) 

N (8) (5) (5) 



Overall 



Mean 0502 0.487 0.439 

SD (0.552) (0.563) (0.524) 

N (47) (38) (38) 
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Table 1 shows 47 knowledge, performance, retention, and time to complete 
measures averaged to provide an overall measure of merit for interactive videodisc 
instruction compared to conventional instruction. Of these, 38 were positive. The effect 
sizes ranged from -1 . 1 4 to 2. 1 8. Their average is 0.50, and their median value is 0.26 for 
effect sizes based on pooled standard deviations and 0.28 for effect sizes based on control 
group standard deviations. Their average of 0.50 is 20-80 percent larger than effect sizes 
found in meta-analyses of computer-based instruction-such as those summarized by 
Niemiec and Walberg (1987). 

Although effect size appears to be an excellent metric for aggregating the results of 
many different studies, its implications for the practical world of instruction are not 
immediately evident A mean effect size of 0.50 from a sample as large as 47 suggests that 
through the use of interactive videodisc technology, a trainer can expect to increase the 
achievement of students by about one-half a standard deviation above their present level of 
achievement. Or, assuming a normal or Gaussian distribution, the average graduate, the 
graduate at the 50th percentile, would improve to the level of graduates currently at the 69th 
percentile through the use of interactive videodisc technology. Military training is more 
concerned with numbers of students who reach defined thresholds of knowledge or 
performance per constant expenditure, than with increases in their achievement. In less 
normative terms then, and assuming a linear rate of learning, an effect size of 0.50 
suggests, roughly, an improvement of 19 percent in number of graduates through the 
application of interactive videodisc instruction. 

Performance and knowledge outcomes are reported separately in Table 1. These 
results show that the average effect sizes for these two achievement measures are similar, 
and suggest that there is little reason to expect interactive videodisc instruction to be any 
more effective for performance outcomes than for knowledge outcomes, although in both 
cases it is more effective than conventional instruction. The average effect sizes for 
performance outcomes range from 0.334 for 15 studies based on control group standard 
deviations to 0.398 for the same studies based on pooled standard deviations. The average 
effect size for knowledge outcomes range from 0.361 for 23 studies based on control 
group standard deviations to 0.375 for the same studies based on pooled standard 
deviations. These results are generally the same for both knowledge and performance 
outcomes. They suggest an improvement of 50th percentile students to about 64th 
percentile achievement or an increase of about 14 percent in the number of students 
reaching criterion achievement levels. 
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Average effect sizes for time to complete instruction-or to reach criterion level(s) of 
achievement-are also shown in Table 1. These averages range from 1.178 for 8 effect 
sizes calculated across all available studies to 1.267 for 5 effect sizes based on pooled 
sta nd ard deviations. These results suggest an improvement of 50th percentile students to 
about the 88th percentile achievement, or an increase of 38 percent in the number of 
students reaching criterion achievement levels if time were held constant. Student time 
savings of about 30 percent have been noted in analyses of computer based instruction 
starting with a widely noted Orlansky and String (1977) study and continuing on into the 
present The average percent time saved across the 8 studies in this review that reported 
time savings was 31 percent. Eight studies do not provide a conclusive base, but these 
results suggest that instruction time savings for interactive videodisc instruction are about 
the same as those for computer-based instruction. 

One might well ask to what degree these time savings were due to self -pacing, to 
what degree they were due to self-pacing uniquely made possible by interactive videodisc 
instruction, and to what degree self-pacing was encouraged in the convention&l instruction 
with which the videodisc instruction in these studies was compared. These questions were 
not well addressed by the studies summarized in this review. Generally the treatments 
were designed to allow the same flexibilities for completing conventional instruction as for 
interactive videodisc instruction, and results from these studies must stand until more 
focused research is performed and reported. 

2. Instructional Settings 

Overall average effect sizes for all four measures of merit were broken out of Table 
1 and calculated separately for military, industrial, and higher education settings. They are 
shown in Table 2. The average effect size for military applications ranges from 0.385, 
based on 20 studies calculated using control group standard deviations to 0.416 based on 
the same experimental comparisons but calculated using pooled standard deviations. These 
results suggest an improvement of 50th percentile students to about 65th percentile 
achievement or an increase of 15 percent in the number of students reaching criterion 
achievement, assuming linear learning. With over 250,000 students involved in 
residential, formal instruction programs each day, these findings promise significant 
improvements for Defense training provided that the operations of schools are changed to 
take advantage of these improvements. 
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The average effect sizes for interactive videodisc instruction used in higher 
education are considerably greater than those for military training. In Table 2, they range 
from 0.660 for 12 studies based on control group standard deviations to 0.725 based on 
the same experimental comparisons but calculated using pooled standard deviations. These 

Table 2. Average Effect Sizes for Military Training, Industrial Training, and 

Higher Education 



Military Training 

Mean 

so 

N 



ES Based on 
Rooted SO 



0.391 
(0.543) 
(24) 



ES Bleed on 
Pooled SD 
When Ctrl SD 

Aten Ai/flilahlo 



0.416 
(0.543) 
(20) 



ES Based 



0.385 
(0.451) 
(20) 



Industrial Traininn 

Mean 0.510 

SO (0.744) 
N (9) 



0.249 
(0.799) 
(6) 



0.174 
(0.838) 
(6) 



Hlohar Friuratlnn 

Mean 0.689 

SO (0.395) 
N (14) 



0.725 
(0.408) 
(12) 



0 .660 
(0.393) 
(12) 



All Settings r>nmhini»rt 

Mean 0.502 

SO (0.552) 
N (47) 



0.487 
(0.563) 
(38) 



0.439 
(0.524) 
(38) 



results suggest an improvement of 50th percentile students to about 75th percentile 
achievement or an increase of 25 percent in the number of students reaching criterion 
achievement Since the emphasis in training is properly on reducing costs to reach 
threshold levels of achievement, this emphasis may actually be observed in practice. The 
lower effect sizes found for military training may be due to this observance-students in 
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training who reach threshold criteria art sent on to new assignments, whereas students in 
education are encouraged to continue improving. If this argument holds* we should expect 
to find effect sizes for industrial training that are closer to those of military training than to 
those of higher education. 

Unfortunately, the findings for industrial training are less stable than those for 
either military training or higher education. In Table 2, they range from 0. 174 for 6 studies 
based on control group standard deviations to 0.510 for 9 studies based on pooled standard 
deviations. The improvement of 50th percentile students suggested by these results ranges 
from 57th to 69th percentile achievement or an increase in the number of students reaching 
criterion achievement levels of 7-19 percent. The three additional studies that are included 
in the effect size average shown in Column 1, are all based on time to complete training. 
The remaining 6 studies, upon which the Column 2 and Column 3 effect size averages are 
based, are either knowledge or performance outcomes measured when the students 
completed the instruction. 

The pattern of knowledge and performance measures across all three instructional 
settings is therefore of interest and is reported in Table 3. As this table suggests, the effect 
sizes from industrial training appear much closer to those of military training than to those 
of higher education if we limit consideration, as Table 3 does, to measures of performance 
and knowledge taken at the end of instruction. 

3. Instructional Approaches 

Another exploration concerned the influence of instructional approach-how the 
videodisc system was uscd-on instructional outcomes. The studies were categorized into 
three groups based on their instructional approach: studies in which the instructional 
approach was directive and tutorial; studies in which the instructional approach was based 
on freely explored simulations; and studies in which tutorial and simulation approaches 
were combined. These studies were crossed with those in which the instructional outcome 
assessed was knowledge or performance, which yielded the four (tutorial, combined, 
simulation, all three) by three (knowledge, performance, combined knowledge and 
performance) set of results shown in Table 4. Not surprisingly, there are too few studies 
in some of these categories. For instance, there was only one study in which a tutorial 
approach was used to produce a performance outcome. It should be emphasized that the 
categorization of instructional approaches in this review was generally based on evaluation 
documentation and not on direct experience with the instructional materials themselves. 
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Table 3. Average Effect 81ms for Combined Knowledge and Performance 
Outcomes In Military Training, Induatrlal Training, and Higher Education 



ES Based on 
Pooled SD 

ES Based on When Ctrl SO ES Based 
Pooled SD AboAyajtobje on Ctrl SD 

Miltarv Trairinn 

Mean 0.353 0.380 0.357 

SD (0.541) (0.533) (0.446) 
N (23) (19) (19) 



Industrial Training 

Mean 
SD 

N (6) (6) (6) 



Mean 0.249 0.249 0.174 

SD (0.744} (0.799) (0.838) 



dtebfll Education 

Mean 
SD 

N (13) (11) (11) 



Mean 0.582 0.601 0.554 

SD (0.354) (0.373) (0.355) 



All Settings Comhinftri 

Mean 

SD 

N (42) (36) (36) 



Mean 0.409 0.464 0.388 

SD (0.536) (0.541) (0.507) 



There appear to be two main results from this analysis. First, there seems to be 
about the same amount of improvement in knowledge and performance outcomes for all 
approaches. There is some mine; evidence here suggesting that simulation either alone or 
in combination with a tutorial yields stronger performance outcomes than knowledge 
outcomes. This finding seems intuitively reasonable, but it is too weak to do anything but 
suggest a line of follow-on investigation. 

Second, there is an obvious increase in effectiveness across these studies from 
simulation-only approaches, which were least effective, through approaches combining 
simulation and tutorials, to tutorial approaches, which were most effective. Because of 
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Table 4. Average Effect 8Ixm for Tutorial Approaches and 
instructional Outcomsa 





Instruc- 
tional 
Approach 


Measure 


ES Based on 
Pooled SD 


CO DMAil 0k n 

co oaseo on 
Pooled SD 
When Ctrl SD 
AtooAvaJable 


ES Based 
on Ctrl SD 




Tutorial 


Moan 


0.703 


0.703 


0.660 












(0 234) 






N 


(6) 


(6) 


(6) 




Combined 


Mean 


0.398 


0.360 


0.385 




Tutorial & 


SD 


(0.420) 


(0.452) 


(0.472) 




oirnu union 


kl 










Simulation 


Mean 


0.104 


0.102 


0.033 






SD 


(0.536) 


(0.604) 


(0.629) 






N 


(9) 


(7) 


(7) 




All 


Mean 


0.361 


0375 


0.361 




Approaches 


SD 


(0.493) 


(0.506) 


(0.533) 




N 


(27) 




(23) 


Performance 


Tutorial 


Mean 


0.805 


0305 


0.805 


Outcomes 




SD 


(na) 


(na) 


(na) 






Kl 


Ml 
I 1 ) 


PJ 


I 1 ) 




Combined 


Mean 


0.423 


0.447 


0.341 




Tutorial & 


SD 


(0.692) 


(0.746) 


(0.554) 




oimuiatton 


IM 


I 11 ) 


Ml 


W 




Simulation 


Mean 


0.191 


0265 


0.250 






SD 


(0.299) 


(0.261) 


(0.284) 






N 


(8) 


(6) 


(6) 




All 


Mean 


0.349 


0398 


0.334 




Approaches 


SD 


(0.556) 


(0.569) 


(0.447) 






N 


(20) 


(15) 


(15) 


jmhined 


Tutorial 


Mean 


0.718 


0.718 


0.678 


Knowledge 




SD 


|A *A At 

(0.230) 


(0.230) 


(0.219) 


A Performance 




N 


(7) 


(7) 


(7) 


Outcomes 














Combined 


Mean 


0.410 


0398 


0.365 




Tutorial & 


SD 


(0.553) 


(0.583) 


(0.495) 




Simulation 


N 


(23) 


(18) 


(18) 




Simulation 


Mean 


0.145 


0.177 


0.133 






SD 


(0.430) 


(0.467) 


(0.494) 






N 


(17) 


(13) 


(13) 




All 


Mean 


0.409 


0.464 


0.388 




Approaches 


SD 


(0.536) 


(0.541) 


(0.507) 






N 


(47) 


(38) 


(38) 
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cost consideration*., jiert is little here that argues against use of simulation. The cost 
arguments for simulation are very strong when a few thousand dollars of software and 
hardware can substitute for actual equipment or field experiences that may cost tens or 
hundreds of thousands of dollars. Results from this review show that such substitutions 
are both feasible and desirable. Cost arguments for tutorial approaches are seldom this 
strong since tutorials can usually be presented by relatively inexpensive means such as 
textbooks, programmed textbooks, or teaching assistants as well as by videodisc hardware. 
Moreover, simulation only approaches showed average effect sizes ranging from 0.C33 
(for knowledge outcomes) to 0.265 (for performance outcomes). They were, overall, 
more effective than conventional approaches. These results, combined with the favorable 
cost ratios that are generally found in simulation approaches, suggest that videodisc 
instruction using simulation alone may turn out to be the most cost-effective alternative 
even though it may be the least effective of the three approaches examined here. 

The greater effect sizes observed for tutorial approaches over combined tutorial and 
simulation approaches are puzzling for at least two reasons. First, it seems intuitively 
reasonable that including simulation with a tutorial should add both value to the tutorial and 
effectiveness to the instruction. Just the opposite is suggested by these data. Second, the 
programmed instruction approach used in tutorial instruction is typically reported by 
reviews of paper-based (programmed textbook) approaches to produce substantially lower 
effect sizes than those found here. For instance, Kulik, Cohen, and Ebeling (1980) found 
that 57 studies of programmed instruction used in higher education to present a variety of 
subject areas improved performance by an average 0.24 standard deviations over 
conventional instruction, which is considerably less than the effect sizes for tutorial 
approaches reported in this review. The effectiveness of tutorial approaches used in 
interactive videodisc instr . t^.. evidently deserves further investigation. 

The finding thru directive, tutorial approaches yield greater instructional results than 
simulation approaches is supported by Gay (1986) who used a 2-by-2 experimental design 
and interactive videodisc instruction to investigate the effects of learner and program control 
on the achievement of students who had high and low prior knowledge of course content. 
Students in the learner control (non-tutorial) condition with low prior knowledge scored 
significantly lower in a post-tesf of course content than did students in the other three 
treatment groups. Gay's results along with those of this review suggest that directive, 
tutorial approaches may be best used when standardized results are to b*. obtained from 
students with a variety of prior knowledge, which is the typical case in military training. 
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On the other hand, high prior knowledge students in Gay's study who were in the 
learner control condition took significantly less time to complete the course and the post-test 
than did students in any of the other three treatment groups. This result may explain some 
of the mixed findings for directive, tutorial approaches found by this review, and it 
suggests that important efficiencies in training may result if the training approach can be 
adjusted for prior knowledge of course content 

4. Level of Interactivity 

Average effect sizes from five within-videodisc comparisons (i.e., those that 
compare high levels of videodisc interactivity with lower ones) are shown in Table 5. 
These range from 0.456 for effect sizes based on control group standard deviations to 
0.397 for effect sizes based on pooled standard deviations. The individual effect sizes that 
contribute to the averages reported in Table 5 range from a single negative effect size of - 
-0.160 to 1.425. Since there are no absolute levels of interactivity-one study's high level 
of interactivity might be another study's low level of interactivity-it is difficult to interpret 
the magnitudes of these effect sizes directly. However, these studies suggest that the 
greater the level of interactivity, the greater the instructional achievement provided by the 
videodisc instruction. 

Table 5. Average Effect Sizes for High Compared with Low Levels of 

Videodisc Interactivity 



Measures 


ES Based on 
Pooled SO 


ES Based on 
Pooled SD 
When Ctrl SD 
Also Available 


ES Based 
on Ctrl SD 


Mean 


0.397 


0.397 


0.456 


SO 


(0.533) 


(0.533) 


(0.631) 


N 


(5) 


(5) 


(5) 



Notably, the Table 5 effect size » based on pooled standard deviations are less than 
the effect sizes based on control group standard deviations. This result is contrary to 
almost all the previous effect size results. It means that the standard deviations across the 
five control groups were smaller than those of the experimental groups, and it suggests that 
when different levels of interactivity are compared within interactive videodisc approaches, 
greater interactivity is associated with greater instructional outcome variability. A result 
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based on only five studies cannot be viewed as conclusive, but it does suggest a variable to 
be considered in future research. 

Four studies of Level II videodisc instruction appeared in our search for empirical 
evaluations. These studies are summarized in Appendix B. As the Appendix shows, 
results from these evaluations are generally less positive than those reported for Level III 
applications. For that matter, five of the seven results that were statistically significant 
were negative, showing the videodisc application to be less effective than conventional 
instruction. Of course the lesser effectiveness of the videodisc instruction may be balanced 
out by lower cost Unfortunately, none of these studies reported cost ratios, and the cost 
effectiveness of these applications remains unknown. 

Table 8 summarizes the average effect sizes for Level II evaluations for knowledge, 
performance, and time to completion measures. Performance outcomes appear much lower 
than knowledge outcomes among these effect sizes, ranging from- 1.752 and- 1.565 for six 
performance outcomes to 0.385 and 0.350 for two knowledge outcomes. There are too 
few studies here to view this as a reliable result Superiority of knowledge outcomes to 
performance outcomes from Level II videodisc presentations must remain for the present an 
interesting possibility. 

The overall averages in Table 8 range from -1.031 for an average of 10 effect sizes 
based on control group standard deviations to-0.702 for 13 effect sizes based on pooled 
standard deviations. They are large enough to be considered seriously. Across the studies 
reported here, videodisc instruction based on Level II technology appears much less 
effective than conventional instruction. 

The Table 6 ; vcrage effect sizes based on pooled standard deviations are smaller in 
absolute magnitude than those based on control group standard deviations. This result 
suggests that there was gener r} y less variation in the conventional instruction groups than 
in the Level II videodisc instruction groups. Since the videodisc instruction groups 
generally achieved lower scores on the outcome measures considered in the Level II 
studies, it may be that some students simply did not adapt to the videodisc presentations 
and found themselves unable to learn effectively from videodisc approaches-whereas other 
students progressed just as well using videodiscs as they would in conventional 
instruction. This possibility could be investigated by examining the raw data from these 
studies. 
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TaWo 6. Avarage Effect Sliee for Knowledge Outcomtt, Performance 
Outcomes, and Inatruetlon Complatlon Times for Inat ruction 
Ualng Uval II Vldaodlaea 



ES Based on 
Footed 80 

ESBaaodon Whan Ctrl SD ESBased 
Pooled SD Also Available on Ctrl SD 

KDOMfJldOt 

Outcamaa 

Maan 0.115 0.350 0.385 

SD (0.276) (0.057) (0.120) 

N (4) (2) (2) 

EflgPUDlDCfl 
QUtQMDM 

Maan -1.565 -1.565 -1.752 

SD (1.331) (1-331) (1.526) 

N (6) (6) (6) 

Tlmw to Complete 
IrwtiueHon 

Mean -0.067 -0.245 -0285 

SD (0.330) (0.163) (0.163) 

N (3) (2) (2) 

Ova rail 

Maan -0.702 -0.918 -1.031 

SD (-1.213) (1.314) (1.488) 

N (13) (10) (10) 



The lower levels of interactivity provided by Level II videodisc systems may 
account for the lower levels of effectiveness found in these studies. Although this is not 
the only explanation, interactivity is one of the main differences between the studies 
summarized in Appendix B and those summarized in Appendix A. Notably, six of these 
results came from the Holmgren et al. (1979) study in which Army Training Extension 
courses designed to be presented using audio tape and film slides were directly recorded on 
videodisc. The findings of this study suggest that tape-slide materials presented on a tape- 
slide medium will produce training results that are superior to those obtained when the 
same materials are presented using another medium. 
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These findings further suggest that the less the unique capabilities of interactive 
videodiscs are used, the less the advantage gained from using the medium. This possibility 
is supported by comparison of the Wankel (1984) with the Stevens (1984) results. The 
saaae basic videodisc material ("Puzzle of the Tacoma Bridge Collapse") was used in both 
cases, and in both cases the same test of physics knowledge was used. When a Level II 
approach was used, the effect size was-0.17; when a Level in approach was used, the 
effect size was 0.29. This possibility is also supported by Verano (19&7), who presented 
three levels of interactivity (linear, segmented, and interactive) to students. The higher the 
level of interactivity he used, the greater the students' achievement. Instructional evaluation 
generally requires a sizable body of research to achieve a conclusive result, but the 
combination of the Hohngren, Wankel, Stevens, and Verano results as well as the contrast 
between Level II and Level m effect sizes make a strong case for using the full capabilities 
of Level m videodisc media in instructional applications. 

Extension of this conclusion to Level IV videodisc materials is probably not 
warranted. The step from Level III to Level IV, under the definitions used for this review, 
only marks a change in the way instructional materials are stored and not any improvement 
in the way they are presented. For that matter, the proprietary nature of all current 
approaches to encoding and decoding digital information on and off videodiscs coupled 
with the lack of additional instructional capabilities makes Level IV videodisc technology 
incompatible with current Defense initiatives to achieve courseware portability (Fletcher, 
1988). 

5. Evaluation Issues 

The above findings suggest favorable results for Level III interactive videodisc 
instruction relative to conventional instruction. They arise directly from available 
evaluation data on the effects of interactive videodisc applications in training and education. 
However, there are two issues to note concerning the studies from which these results are 
drawn: 

(1) Most of the evaluations reported here were performed by developers who also 
produced the interactive videodisc instruction being evaluated. Few of the 
evaluations were performed by third parties. There are strengths and 
weaknesses in this approach. Developers are rarely indifferent to the success 
of their products and may, intentionally or not, bias the results of their 
evaluation. On the other hand, there are arguments that support initial 
evaluation of instructional materials by its developers and not by a third party 
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evaluator. For example, there are standards for the performance and 
documentation of evaluation studies. Observance of these standards can be 
assessed by others~~as they were in the search for evaluation studies 
summarized here— and bias in studies performed by developers and selected for 
this review should be minimal. Also, developers have a stake in an honest 
assessment of their products. The better the information they get, the better 
their practice and products can become. Finally, developers may be in the best 
position to evaluate their products. They may understand better than anyone- 
including potential users-the strengths and limitations of what they have 
developed, and they are better prepared than anyone to assess their products. 

(2) One difficulty for any evaluation of an emerging technology is that there is 
nothing else like it. Each new technology has its own strengths and 
limitations. If the evaluation is held to strict instructional and experimental 
controls based on the older technology, the newer technology will be at a 
disadvantage. This problem can be ameliorated somewhat by focusing on 
instructional outcomes and not on what is done to accomplish these outcomes. 
Even in the best of situations, however, new approaches are unlikely to be 
used well since not enough is understood about how best to employ them. 
Despite all the evaluations listed, we may have yet to see an approach to 
instruction that uses interactive videodisc technology to best advantage. 

In sum, it can be argued either that the view of interactive videodisc applications 
seen here is overly optimistic or overly pessimistic. At present it seems best to rely on the 
data we have and to conclude that interactive videodisc instruction offers genuine and 
demonstrable gains in instructional effectiveness over the conventional approaches we use 
now, ev n though we understand little about how these gains come about 

C. COST 

Effectiveness by itself is only half the story. Military trainers, along with their 
civilian counterparts, are more frequently asking what their investment in instructional 
technology has bought in increased instructional effectiveness. Cost ratios (calculated as 
the ratio of experimental treatment costs over control treatment costs) were used to assess 
costs. Ratios in this investigation that are less than 1.0 indicate lower costs for interactive 
videodisc instruction than for conventional insuuction. 
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Thirteen cost ratios arc reported in Appendix A. All of these ratios are less than 
1.0. Most came from studies concerned with training applications. Six of the studies 
involving military training reported cost ratios (Wilkinson, 1983; Gcchinelli et al., 1984; 
Pieper et al., 1984; Vernon, 1984; Green et al., 1986; and Winkler and Polich, in press); 
one of the studies (Aukerman, 1986) involving industrial training did; and one of the 
studies (Davis, 1985) involving higher education did Five of the cost ratios reported art 
for initial investment U), four are for operating and support costs (OS), and four for some 
combination of initial investment with operating and support costs. These ratios are all 
calculated as the costs of interactive videodisc instruction over the costs of conventional 
instruction. Since there are too few ratios here to provide reliable averages, the cost 
findings from these eight studies and 13 ratios are briefly summarized below. 

Wilkinson (1983) compared the costs of using actual equipment with simulations of 
the equipment presented by interactive videodisc in electronics maintenance training. 
Hardware costs of the videodisc system were $7,180. Total software production costs 
were $18,700, but costs to duplicate the software were negligible so that the marginal costs 
for the software declined with each videodisc station equipped with it. These costs should 
be compared to the standard cost of $51,000 for each workstation involving actual 
equipment The per unit costs of actual equipment workstations would decrease only 
slightly with increases in quantities purchased. The training course personnel 
recommended that a minimum of eight workstations be provided for student use, and the 
cost ratio of 0.19 reported in Appendix A for initial investment was based on the 
assumption that eight workstations would be purchased. 

Gcchinelli et al. (1984) also compared the costs of usin b ictual equipment versus 
videodisc simulations of the equipment in electronics maintenance training. Their findings 
indicated that the videodisc-based training system would cost slightly more than the training 
system using actual equipment for the first copy but that costs for subsequent copies would 
drop off quickly. The break-even point occurred at about 3 copies. Cicchinelli et al. 
included courseware preparation costs with costs for the videodisc system, but did not 
include courseware costs with costs for the actual equipment training system since 
courseware costs were unavailable. Consequently, the costs for interactive videodisc 
instruction were somewhat inflated relative to the costs for instruction using actual 
equipment 

Cicchinelli et al. considered two "scenarios" in comparing the costs of simulations 
using an interactive videodisc system in piace of actual equipment. In one scenario they 
assumed that 10 actual equipment trainers were already in place and that an additional 10 
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woe needed The additional trainers could be videodisc systems or actual equipment The 
cost ratio of videodisc systems to older (and less expensive) actual equipment was 0.94 for 
the additional trainers. The ratio was 0.84 for videodisc systems over newer equipment 
actually being installed in operational settings. A second scenario considered the costs of 
modifying 10 videodisc systems to reflect engineering changes in the operational equipment 
versus modifying the actual equipment u'cd for training. Basically the comparison was 
between software changes in the videodisc instruction (15 new lessons, new technical 
documentation, and videodisc remastering) versus hardware changes in the actual 
equipment The ratio of modification costs for videodisc instruction over actual equipment 
instruction in this scenario was 0.44. 

Pieper et al. (1984) presented another study of simulation based on interactive 
videodisc versus actual equipment in electronic maintenance training. The ratio of single 
station initial investment costs for hardware was 0.02 for a videodisc-based approach over 
an actual equipment approach. The similar ratio for continuing hardware costs was also 
about 0.02. The overall ratio for initial investment costs including hardware, software, 
documentation, and administration was 0.27 for a videodisc-based approach over an actual 
equipment approach. The similar overall ratio for continuing costs was 0.03. 

Vernon (1984) compared the effectiveness of simulation using interactive videodisc 
instruction with actual equipment to train soldiers to operate data communications 
terminals. The ratio of initial investment costs including hardware, software, and all 
courseware for interactive videodisc training over actual equipment training was 0. 15. The 
ratio of continuing costs including the amortized initial investment for hardware, software, 
and courseware for interactive videodisc training over actual equipment training was 0.13. 
Considering maintenance costs alone, without the amortizing initial investment costs, the 
cost ratio for interactive videodisc training over actual equipment training was 0.01. 

Green ct al. (1986) discussed the 15-year life cycle costs for 'he approaches studied 
by Williams and Harold (1985), who compared classroom instruction alone with classroom 
instruction supplemented by interactive videodisc materials. Classroom instruction for both 
groups included use of the actual equipment, a sonar set costing $34 million per copy, on 
which students were learning to perform maintenance. The ratio of costs for operating the 
course over a 15-year period, including the initial costs of hardware, software, and 
courseware, was 0.05 for the substitution of about 17 hours of interactive videodisc 
instruction over instruction using actual equipment for that segment of the course. 
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Winkler and Polich (in press) compared instruction combining interactive videodisc 
and actual equipment with instruction using actual equipment only. Costs for actual 
equipment hardware were $138,000 per copy compared with $5,500 per copy for the 
interactive videodisc hardware. The ratio for the overall combination of initial investment 
and operating and support costs for interactive videodisc instruction over the costs for 
instruction using actual equipment was 0.17. 

Aukerman (1986) considered only student and instructor time costs. She found 
costs of about $28.61 per learner for about 2.6 hours of instruction using interactive 
videodisc versus about $46.49 per learner using lectures and discussion yielding a cost 
ratio of 0.61 for interactive videodisc over the more conventional approach. Most of the 
cost differences were due to the substantially reduced amount of instructor time required for 
the interactive videodisc instruction -about 8.5 minutes versus 66 minutes for 
lecture/discussion. 

Davis (1985) compared wet-laboratory facilities with videodisc-based facilities for 
instruction in respiration for biology. She did not include instruction preparation costs in 
her analysis for either of these approaches, which is not unreasonable since these costs may 
be nearly equal or they may be trivial compared to life cycle support costs of both 
instructional approaches, but she did include space preparation, personnel, supplies, and 
equipment costs. Assuming instruction involving three different organisms kept at three 
different temperatures and a need to renovate space for both instructional approaches, her 
cost data indicate a ratio of 0.80 for initial investment and 0.02 for operating and support 

Other studies, not summarized in Appendix A, reported useful analyses comparing 
the costs of interactive videodisc instruction with conventional instruction without 
comparing their instructional effectiveness. Doughty and Lent (1984) present one such 
comparison involving the actual costs of the two approaches. This comparison concerns a 
training facility for a large jet engine rebuilding facility. Doughty and Lent estimated the 
total costs for procuring, installing, and maintaining interactive videodisc instruction for 
this facility over a 10 year period to be $4,419,000. Three sources of major cost 
avoidances to be realized from installation of this interactive videodisc instruction were 
identified as engine failure avoidance, training cost avoidance, and staff savings. Doughty 
and Lent estimated the total 10 year savings from these three sources alone to be 
$14,031,000, yielding a return on investment of 316 percent. 

Many commentators have remarked on the advantages of computer-based 
instruction and interactive videodisc instruction for delivering standardized, decentralized 
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training to studcnts-rathcr than delivering students to the training. Walker (1985) 
presented an industrial training study in which the costs of delivering interactive videodisc 
instruction to remote sites is compared to the costs of centralized training. When the initial 
investment costs for developing and installing the interactive videodisc training are 
amortized over 3 years Walker showed that the costs per student are $1,568 for the 
centralized training and $553 for the interactive videodisc training-a cost ratio of 0.35 for 
combined initial investment and operating and support costs. 

After an extensive study on the feasibility, costs, and effectiveness of various 
methods for training smog-check mechanics, Maher (1988) concluded that because of the 
elimination of instructors and reduced training time, videodisc instruction would provide 
the most cost-effective approach of five that were considered for hands-on mechanic 
training and verification testing. Maher found that the costs for videodisc training would be 
$50.60 per student compared with baseline costs of $102.78 per student-a cost ratio of 
0.49 for combined initial investment and operating and support costs. 

As Orlansky and String (1977) suggested, an important source of cost avoidance in 
many training settings is student time. If the 30 percent savings they reported for students 
to reach threshold levels of performance using computer-based instruction also obtains for 
interactive videodisc instruction, and it was found in this review to be about 31 percent, 
then savings for some high student load courses will reach millions of dollars. A 
potentially greater source of savings suggested by Solomon (1986) would be reductions in 
the number of people the military has to support in a job category if the time needed to train 
people for it can be reduced. This is an intriguing suggestion, but it requires analyses 
beyond the scope of this review. 

D. COST-EFFECTIVENESS 

Interactive videodisc instruction was found to be less costly in nearly all studies that 
reported costs. It was also found to be more effective overall than convention instruction. 
This combination of findings suggests strongly that interactive videodisc instruction may 
prove to be the most cost-effective alternative for instruction in many applications. 
However, this result is best viewed as suggestive rather than as conclusive because none of 
the reviewed studies examined both costs and effectiveness in a properly controlled 
empirical study based on a systematic model of cost inputs and effectiveness outputs. 

This is a significant consideration. There are many different costs that may or may 
not be associated with an instructional approach. Costs reported for any approach will vary 
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widely depending on the assumptions and procedures used to collect-or ignore-these 
different costs. If these assumptions and procedures are not made explicit, as they often 
are not, it is difficult to judge their relevance to the reader's concerns, and it is impossible 
to aggregate them across different studies-as we would like to do in a review such as this— 
and build up a composite understanding of what they might be for an approach such as 
interactive videodisc instruction. 

Budgets are inadequate for cost estimation because they do not include all the 
components needed for an adequate cost accounting; they may distort true costs of a 
component to accord with local accounting practices; and they represent planned, not actual 
expenses. After-the-fact analyses of costs are inadequate for roughly the same reasons, 
and it is especially difficult to disaggregate costs for one purpose (e.g., cost accounting) 
after they have been assembled for another (e.g., production control and auditing). 

Costs and effectiveness data should be drawn from the same comparative study for 
two reasons. First and most obviously, we need to establish the same, proper experimental 
controls for all conditions (i.e., the experimental and control treatments) for which cost and 
effectiveness data will be drawn. Second, we need to ensure that the same cost models are 
used for all treatments and that they are related to the same effectiveness models-which 
may be nothing more than end of instruction knowledge or performance, but which must 
be assessed using measures common to all treatments. These issues are discussed more 
fully by Levin (1983) and Fletcher (1990). 

Nonetheless, it is a clear finding of this review that interactive videodisc instruction 
was observed to be both less costly and more effective across the studies found and 
examined. Although the cost-effectiveness of interactive videodisc instruction can vary 
widely depending on many factors, it appears that there will be many instances in which 
interactive videodisc instruction is the most cost-effective alternative. 

E. TIME ON TASK 

Very little direct evidence exists to indicate whether students using interactive 
videodisc instruction are willing or likely to spend more time in instruction or whether the 
time they do spend is more task centered than it is in conventional instruction. A number of 
studies listed in Appendix A surveyed students* opinions of the interactive videodisc 
instruction they received. By and large these studies report that the students enjoved the 
interactive videodisc instruction, they would recommend it to others, and they found it easy 
to use, all of which suggests that they might spend more time on relevant instructional tasks 
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when they use interactive videodisc instruction. Therefore, it is not unreasonable to expect 
interactive videodisc instruction to increase time on task. 

Winkler and Polich (in press) investigated interactive videodisc instruction used to 
supplement actual equipment experience for military radio operators. They found that 
introduction of the interactive videodisc materials led to a 45 percent increase in the time 
students spent practicing radio installation. These students received increased practice 
without increasing the amount of total time they spent in training. This result is promising, 
but a single study cannot be viewed as conclusive. 

F. RETENTION 

Given the compelling nature of much interactive videodisc instruction, it seems 
reasonable to expect it to be memorable-skills and knowledge obtained under interactive 
videodisc instruction should be retained at least as long and perhaps longer than skills and 
knowledge obtained under other instructional approaches. Four of the studies in Appendix 
A [Young and Tosti (1981), Bundersor- et al. (1984), Verano (1987), and Jones (1988)] 
addressed retention. All four were concerned with the retention of knowledge, and Jones 
also considered the retention of performance skills. Effect size (calculated from pooled 
standard deviations) for the single measure reported by Young and Tosti increased from 
-0.39 to 40. 14 over a retention interval of 10 weeks. Effect sizes (calculated from pooled 
standard deviations) for the two measures reported by Bunderson ct al. reduced from an 
average of about 0.59 to about 0.38 over the 1-week retention interval studied. Effect sizes 
(again calculated from pooled standard deviations) based on results reported by Verano for 
interactive videodisc instruction over "placebo", non-instruction reduced from 2.94 to 1.33 
over the 4-week retention interval studied. In these three studies, the experimental groups 
and the control groups seem to have grown together over the retention interval. Jones 
reported retention data but not end of instruction achievement. Her data indicate substantial 
effect sizes remaining in favor of the interactive videodisc training 2-3 weeks after the 
instruction was completed for both knowledge and performance outcomes. 

Some research suggests that retention is sensitive to the choice and sequence of 
instructional media. Baggett (1983) showed that performance on assembly tasks assessed 
after a 1-week retention interval was significantly improved for students who had hands-on 
practice first and viewed a film second rather than the other way around. More recent 
experimental work by Baggett (1988) has shown no advantage in retention obtained from 
hands-on practice presented at the same time as audiovisual instruction-groups without 
practice performed as well as groups with practice under these conditions. She did show 
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an advantage in both immediate and 1-week delayed performance for practice and 
audiovisual instruction that was presented sequentially and not simultaneously. The best 
r^rUs were again shown in groups who received the hands-on practice first followed by 
ui? iudiovisual instruction and not the other way around This finding seems in keeping 
with the informal observations of experienced i military trainers that interactive videodisc 
instruction achieves better results in advanced training than in beginning training. 
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IV. SUMMARY 



The results of this review and its conclusions principally concern the instructional 
capabilities, or functionalities, made available by interactive videodisc systems. Systems 
other than those that link videodisc players to computers but that provide the same 
instructional functionalities may achieve at least the level of effectiveness observed here. At 
present there is no hardware technology that can competitively provide the functionalities 
that interactive videodisc systems can, but digital video, digital audio, and compact disc 
technology are all developing rapidly and will soon claim at least some of the territory now 
held by interactive videodisc systems. The findings that follow are based on capabilities, 
not hardware, and should obtain for any system that provides the functionalities now found 
in interactive videodisc instruction. 

A. EFFECTIVENESS 

(1) Interactive videodisc instruction was successfully used to teach. The four 
comparisons listed in Appendix A of interactive videodisc instruction with a "placebo" 
treatment in which no relevant instruction was given, resulted in an average effect sire 
calculated using pooled standard deviations of 1.39 (an increase of 50th percentile students 
to about 92nd per ^entile achievement) and an average effect size calculated using control 
group standard deviations of l.SS (an increase to about 95th percentile achievement). 

(2) Interactive videodisc instruction was more effective than conventional 
approaches to instruction. Forty-seven comparisons of interactive videodisc instruction 
with conventional approaches were identified for this review. As shown in Table 1 above, 
the average effect size for these 47 comparisons, calculated using pooled standard 
deviations over all instructional settings, instructional approaches, and outcomes, was 
found to be 0.50 (an increase for 50th percentile students to about the 69th percentile of 
achievement). 

(3) Interactive videodisc instruction was more effective than entional 
instruction in military training. Twenty-four comparisons of inter e videodisc 
instruction with conventional approaches to military training for subjc* ~as that included 
maintenance, operator, and command training were identified for this isview. As shown in 
Table 2 above, the average effect size for these 24 comparisons, calculated using pooled 
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standard deviations over all instructional approaches and outcomes, was found to be 0.38 
(an increase for 50th percentile students to about the 65th percentile of achievement). 

(4) Interactive videodisc instruction was more effective than conventional 
instruction in higher education. Fourteen comparisons of interactive videodisc instruction 
with conventional approaches to instruction in higher education settings were identified for 
this review. Over all instructional approaches and outcomes, there was an average increase 
in achievement calculated using pooled standard deviations of 0.69 standard deviations (an 
increase for 50th percentile students to about the 75th percentile of achievement). 

The lower average effect sizes found for military training than for higher education 
may be due to a focus in military training on reaching threshold level(s) of achievement 
with minimum cost and time so that students who reach achievement criteria arc sent on to 
duty assignments rather than held in the instructional setting as they are in higher education. 
This practice contrasts with the focus in education on maximizing achievement while 
holding costs constant so that all students remain in instruction for roughly the same 
amount of time allowing more efficient instructional approaches to yield higher levels of 
achievement. This possibility is supported by the effect sizes found for industrial training, 
which may be managed in ways more closely resembling military training than higher 
education. The industrial training results shown in Table 2 vary widely. However, when 
the effect sizes arc limited to knowledge and performance outcomes alone, as they are in 
Table 3 above, the results from military and industrial training are much closer together and 
both differ markedly from higher education. 

(5) Interactive videodisc instruction was equally effective for both knowledge and 
performance outcomes. The average effect sizes calculated using pooled standard 
deviations for 27 knowledge outcomes (facts, concepts, and other information students 
acquired) and for 20 performance outcomes (procedures, skills, and other capabilities 
students could demonstrate) both averaged around 0.35 standard deviations (suggesting an 
improvement of 50th percentile students to about the 64th percentile). 

(6) The more the interactive features of interactive videodisc technology were 
used, the more effective the resulting instruction. This issue was examined directly by 
Verano (1987) who compared interactive videodisc instruction "4th segmented videodisc 
instruction and linear videodisc instruction and found consistently greater achievement with 
greater levels of interactivity. The issue can also be examined by comparing the overall 
effect sizes for Level II videodisc instruction with those found for Level III videodisc 
instruction. Table 6 (above) shows an average effect size of -0.70 calculated using pooled 
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standard deviations for 13 results from Level II videodisc instruction, which contrasts with 
the overall effect size of 0.50 found for 47 results for Level III videodisc instruction (Table 
1). Effects of interactivity can also be examined by a comparison of the results from 
Wankers (1984) study (Appendix B) with those from Stevens's (1984) study (Appendix 
A). Both studies used the same videodisc materials, and both tested for physics knowledge 
outcomes. Wankel's study used Level II videodisc materials and produced an effect size of 
-0,17, Stevens's study used Level III videodisc materials and produced an effect size of 
0,29. The only exception to this pattern of greater achievement with greater interactivity 
occurred in one of Fowler's (1980) two comparisons of branched interactive videodisc 
instruction with linear videodisc instruction (Appendix A). This comparison resulted in an 
effect size of -0.28 and favored linear videodisc instruction - the other resulted in an effect 
size of 0.72 and favored branched videodisc instruction. Aside from this single result, the 
pattern of greater achievement with greater interactivity was consistent across the studies 
identified for this review. 

(7) Directed, tutorial approaches were more effective than stand-alone simulations 
in interactive videodisc instruction. This issue is addressed by the effect sizes shown in 
Table 4 (above). Similar results were found for knowledge outcomes, performance 
outcomes, and knowledge and performance outcomes combined. The effect sizes averaged 
around 0.70 for tutorial approaches, 0.40 for combined tutorial and simulation approaches, 
and 0.15 for simulations by themselves. Tutorial guidance appears to pay off. This result 
is consistent with a body of literature that suggests that simulators and training devices are 
more effective when incorporated into a complete training system than when they are used 
as stand-alone resources. On the other hand, the cost arguments for simulation over actual 
equipment use are sufficiently cogent that it may still be the most favorable alternative from 
a cost-effectiveness standpoint 

(8) Within-group variability of achievement was smaller in interactive videodisc 
instruction than in conventional instruction. In nearly every comparison of Level III 
interactive videodisc instruction with conventional instruction, the effect size calculated 
using pooled standard deviations was larger than the effect size calculated using control 
group standard deviations. This result suggests that achievement in the interactive 
videodisc groups was less variable and more equitably spread throughout the student 
population than it was in the conventional instruction groups. It has been suggested that 
one benefit of interactive over conventional instruction is that fewer students art left behind 
- interactivity insures that every student receives some individualized attention and is 
therefore either goaded or aided into some progress toward the instructional goals. This 
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suggestion is supported by the findings shown in Tables 1-4. It is also anticipated by 
commentators on individualized instruction and aptitude-treatment interaction (e.g., Corno 
and Snow, 1986). If it explains the pattern of standard deviations, it will be particularly 
important for training settings in which the intention is to bring every student to criterion 
thresholds of performance in contrast to education settings in which maximizing each 
student's performance is more likely to be emphasized. 

(9) The effectiveness of interactive videodisc instruction was greater than 
computer-based instruction has been found to be. Both approaches have been found to be 
more effective than conventional approaches to instruction, but on the basis of studies 
reviewed here, interactive videodisc instruction was the more effective of the two 
approaches. The average effect size of around 0.70 observed for interactive videodisc 
instruction used in colleges is considerably higher than both the average effect size of 0.26 
found by Kulik and Kulik (1986) for computer based instruction in colleges and the 
average effect size of 0.42 found by Kulik, Kulik, and Shwalb (1986) for computer-based 
instruction in adult education. 

Videodisc capabilities increase the hardware costs of a standard computer-ba. xl 
instruction system by about $2,500 and the software or instructional preparation costs by 
widely varying amounts depending on the type and use of visual information in the 
courseware. One might well argue for commensurate increases in the effectiveness of the 
instruction being delivered. However, these are initial investment costs only. The life 
cycle costs of computer-based instruction compared to videodisc instruction may differ 
trivially, if at all. Nonetheless, differences in costs, instructional objectives, and 
instructional approaches across all possible applications make blanket recommendations for 
one or the other of these technologies ill-advised. Recommendations concerning the use of 
computer-based instruction and interactive videodisc instruction in specific applications are 
probably best made on a case-by -case basis. 

(10) It appears reasonable to conclude on the basis of these findings that 
interactive videodisc instruction is a promising approach for both training and education, 
but there was little in the reviewed studies to indicate how interactive videodisc instruction 
achieves its success. Generally, interactivity seemed valuable and more interactivity 
appeared to produce more achievement, but the nature of this relationship (i.e., how much 
of what sorts of interactivity produce what kinds of achievement in what amounts) retrains 
unknown. Generally, more needs to be learned about what and how the different 
instructional features of interactive videodisc instruction contribute to the increased student 
achievement observed across the studies reviewed here. 

IV-4 

O r: m 



Further, there are many outcomes such as speed of response, accuracy of response, 
attitude toward the subject matter, insight, transfer, and retention that either alone or in 
some combination may become the objectives of instruction. How different design 
alternatives for interactive videodisc instruction contribute quantitatively to these outcomes 
(Le., bow to design instruction to achieve specific outcomes) remains poorly understood at 
best Follow-on research might begin with detailed examination of the interactive videodisc 
instruction materials identified by this review. 

B. COST 

Interactive videodisc instruction was found to be less costly than conventional 
instruction in nearly all the studies reviewed here that examined costs. All 13 cost ratios 
(the ratio of costs for interactive videodisc instruction over conventional instruction) 
reported in Appendix A are less than 1.0 indicating lower costs in every measured instance 
for interactive videodisc instruction. The average across these 13 cost ratios is about 0.36. 
Most of the low cost ratios reported in Appendix A are due to the substitution of interactive 
videodisc hardware and simulation software for actual equipment hardware in training. 
These are favorable results for the use of interactive videodisc instruction, but it should be 
noted that they were all calculated from different categories of costs and from a variety of 
cost models. It seems reasonable to conclude that interactive videodisc instruction is less 
costly than conventional instruction in many areas, but that specific costs savings, if any, 
must be determined on a case-by-case basis. 

Another area of cost avoidances involves reductions in time to reach criterion levels 
of achievement in instruction. If students receive pay and allowances while they are in 
school, then student time reductions are an especially important source of cost avoidances. 
The average time saved across the studies covered by this review was 31 percent, which 
matches closely the 30 percent time savings reported by Orlansky and String (1977) for 
computer-based instruction. With over 250,000 students involved in formal, residential 
training each day, these findings promise significant reductions in military training costs. 
The reductions in total military manpower that can result from such time savings ait likely 
to be an additional and significant source of cost savings. 

C. COST-EFFECTIVENESS 

Interactive videodisc instruction was found to be both less costly in all studies that 
reported costs and more effective overall than the conventional approaches with which it 
was compared. This finding should bt viewed as suggestive rather than conclusive since 
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no studies were found that used systematic models to provide empirical data on both cost 
inputs and effectiveness outputs. 

D. TIME ON TASK 

Interactive videodisc instruction may increase time on task. Intuitively, the low cost 
of interactive videodisc instruction relative to the costs of actual equipment should mean 
that instead of passively watching demonstrations on actual equipment, students can be 
actively engaged with equipment simulated by interactive videodisc technology-learning by 
doing, not learning by watching. Only one study reported an observation relative to this 
point Winkler and Polich (in press) reported a 45 percent increase in the time spent 
practicing radio installation as a result of the introduction of interactive videodisc. This 
result seems best viewed as promising, but not conclusive. 

D. RETENTION 

Based on evidence presented here, there is little reason to believe that interactive 
videodisc instruction will have a significant effect on retention of either knowledge or 
performance. The four studies in Appendix A that reported retention data show a drift of 
both positive and negative results back to zero (i.e., to no differences in achievement 
between interactive videodisc instruction and conventional instruction). Post-training 
experiences typically influence retention of knowledge and performance in a powerful 
manner, and they may well wash out any effects of die instructional approach used to teach 
the knowledge or performance in the first place. The findings for interactive videodisc 
instruction in this review corroborate other Defense experiences with new training 
approaches. It has generally been found that their promise and payoff reside primarily in 
improving the efficiency of instruction, not in assuring that what is learned will be retained 
longer. 



9 

ERIC 



IV-6 



V. CONCLUSION 



The 47 studies reviewed here indicate that for many different instructional settings 
and many different instructional objectives, interactive videodisc instruction is both more 
effective and less costly than conventional instruction. They suggest, therefore, that 
interactive videodisc instruction can have a significant positive impact on the productivity of 
resources allocated to military training and education and on the availability of people 
needed to operate and maintain military systems. Although more needs to be learned about 
how interactive videodisc instruction should be designed and employed, it should now be 
routinely considered and used in military training and education. 
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APPENDIX A 
STUDIES OF LEVEL III VIDEODISC INSTRUCTION 
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Notes Concerning Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure; 
P - Performance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX - Peformance 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I -- Interactivity Comparison; DC -- 
Interactivity Average; B -- Baseline Comparison; BX - Baseline Average 
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Notes Concerning Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure; 
P - Performance Measure; R - Retention; D - Time to Complete Instruction; KX Knowledge Average; PX - Peformance 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I - Interactivity Comparison; IX - 
Interactivity Average; B Baseline Comparison; BX - Baseline Average 
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Table A-l (Continued). Studies of Level m Videodisc 
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Notes Concerning Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K -- Knowledge Measure; 
P - Performance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX - Peformance 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, A D Measures); I - Interactivity Comparison; IX 
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Notes Concerning Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K -- Knowledge Measure; 
P - Performance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX - Peformance 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I - Interactivity Comparison; DC - 
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Notes Concerning Effect Size: 
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Notes Concerning Effect Size: 

T ~ Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure; 
P - Performance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX - Peformance 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I - Interactivity Comparison; IX - 
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^2 Interactivity Average; B ~ Baseline Comparison; BX ~ Baseline Average 
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Table A- 1 (Continued). Studies of Lev el m Videodisc Instruction 



Instructional Instructional 
SttliQg Content 



Instructional 
Approach N 



Comparison Outcome 



ES ES % Cost 
Comb SD Ctrl SD Imor. Ratio Reference 



lSExp IVDvs 
14Ctrll Role Play vs 
15Cni2 Prog. Text 

IVDvs 
Role Play 

IVDvs 
Prog. Text 



14Exp IVDvs 
16Ctrll Role Play vs 
15Ctrl2 Prog. Text 

IVDvs 
Role Play 

IVDvs 
Prog. Text 



14Exp IVDvs 
13Ctrll Role Play vs 
14Ctrl2 Prog. Text 

IVDvs 
Role Play 



Content Test4 ANOVA 
(Performance ns 
Counseling) 

-.16 CK 
ns 

.69 CK 
ns 

.265° 

Content Test5 ANOVA 
(Insubordina- p<.05 
tion) 

.88 CK 
p<.05 

.78 CK 
p<.05 

Content Test6 ANOVA 
(Personal ns 
Crisis) 

-.46 CK 
ns 



-.17 -6 



.74 33 



.285 



1.12 43 



.97 38 



1.045 



-.45 -12 



Notes Concerning Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure; 
P Performance Measure; R - Retention; D - Time to Complete Instruction; KX -- Knowledge Average; PX - Peformance 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I - Interactivity Comparison; IX 
Interactivity Average; B- Baseline Comparison; BX- Baseline Average 
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Tabic A-l (Continued). Studies of Level m Videodisc Instruction 

Com 



Instructional 
Setting 



Military 
Training 



Military 
Training 



Instructional 
Content 



Instructional 
Approach fcL 



S2Q QutWTOC 



Helicopter 

Maintenance 

(Hydraulics) 



Electronic 

Equipment 

Operation 



Tutorial on 36Exp 
Simulated 35Ctrl 
Equipment 



Tutorial with 169Ctrl 
Simulated 167Exp 
and Actual 
Equipment 



IVDvs 
Prog. Text 



Compulsory 
Independent 
Study with vs 
without IVD 



IVD with Actual 
Equipment vs 
Actual 

Equipment Only 



ES ES % Cost 
Comb SD Ctrl SD Impr. Ratio Reference 



-.18 C - K 
ns 



Paper Test of ,48 C K 
of Course p<.05 
Knowledge 

Performance ,14 c,p 
Test Using ns 
Training 
Device 

.31^ 

Performance -03 CP 
Test (IF Gain ns 
Alignment) 
Using AE 



Performance 
Test (AGC 
Alignment) 
Using AE 



.09 cp 



ns 



Ptrformance -,09 c,p 
Test (Squelch ns 
Adjustment) 
Using AE 



-.16 -6 



-32 ICX -.305 



na 



na 



na 



-.03 -4 



.09 11 



-.08 -4 



na 



Malec and 

Luszczak 

(1987) 



.17 Winkler and 
(II Polich 
&OS) (In press) 



Notes Concerning Effect Size: 

T - Tutorial Approach; C ~ Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure; 
P - Performance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average, PX - Peformance 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I - Interactivity Comparison; IX - 
Interactivity Average; B - Baseline Comparison; BX - Baseline Average 
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Table A-l (Continued). Studies of I^vcl III Videodisc Instmcrinn 
Instructional Instructional Instructional 



ES ES % Cost 
Outcome CombSD CtrlS D Iirmr. Rajjo, BsfcgQa 



Knowledge -.07 CK 
Test (Paper) ns 



-.08 -1 



-.01 px -.01 



2 11 Ctrl IVD with Actual 
217Exp Equipment vs 
Actual 

Euipment Only 



Industrial 
Training 



Electronics 
Maintenance 



Tutorial with 15Exp 
Simulated 14Cui 
Equipment 



Performance 
Test Errors 
Using AE 
Simulator 

Time to 
Complete 
Performance 
Test Using 
AE Simulator 

Trials to 
Complete 
Performance 
Test Using 
AE Simulator 



IVD vs Time to 

Linear Videotape Complete 
& Workbook Course A 



-.025 AX 

.18 c - p 
ns 



.14 c - p 
ns 



.21 c - p 
P<.05 



.18 PX 

1.03 CD 
p<.001 



-.025 
.19 14 



.13 7 



.18 18 



.17 

na 35 



na 



na Spencfci 
(1983) 
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Notes Concerning Effect Size: 



T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure- 
P - Performance Measure; R -- Retention; D - Time to Complete Instruction; KX Knowledge Average; PX -- Peformance ' 
Average; RX - Retention Average; AX - Overall Average (for K, P, & D Measures); I - Interactivity Comparison; IX -- 
Interactivity Average; B ~ Baseline Comparison; BX - Baseline Average 
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Table A- 1 (Continued). Studies of Level m Videodisc Instruction. 

C£ 



Instructional Instructional 
Siting Content 



Instructional 
Approach N_ 



ES ES % Cost 
Outcome Comb SD Ctr l SD Imor. Rario Reference 



7Exp 
16Ctrl 



13Exp 
12Ctrl 



Tune to 
Complete 
Course B 

Tune to 
Complete 
Course C 



94CD 
p<.05 

1.12 C - D 
p<.001 



na 



na 



23 



47 



na 



na 



Also in 
May (1984) 



Industrial 
Training 



CPR Recertifi- Simulation 15Exp 
cation Training and Sensoied 14Ctrl 
for Registered Manikin 
Nurses 

14Exp 
14Ctrl 



15Exp 
14Ctrl 



14Exp 
13Ctrl 



IVDvs 
Lecture 
(Acute Care) 



Knowledge 
Test 



IVD vs Knowledge 
Lecture Test 
(Non-Acute Care) 



IVDvs 
Lecture 
(Acute Care) 

IVDvs 
Lecture 

(Non-Acute Care) 



Skill Test 



Skill Test 



-.90 s * 
ns 

.OO^ 
ns 

-.02 s - p 
ns 

07S.P 
ns 



-1.14 -4 



.00 0 



-.07 -1 



.07 1 



.61 Aukerman 
(OS) (1986) 



Industrial Communication Tutorial with lOExp IVD(Exp)vs 
Training Circuit Equipment lOCtrll aassroom(Ctrll) 

Maintenance Simulation 12Ctrl2 vs On-Job Tmg 

(Ctrl2) 



na North (1988) 



Notes Concerning Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure; 
p - Performance Measure; R - Retention; D - Time to Complete Instruction; KX ~ Knowledge Average; PX - Peformance 
Average; RX - Retention Average; AX - Overall Average ;for K, P, R, & D Measures), I ~ Interactivity Comparison; IX - 
Interactivity Average; B - Baseline Comparison; BX - Baseline Average 
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Tabic A-l (Continued). Studies of Level m Videodisc Instr»rrinn 



Instructional 
Setting 



Instructional 
Content 



Instructional 
Approach £L 



son Outcome 



Industrial 
Training 



Higher 
Education 



Handling of 

Hazardous 

Materials 

Equipment 
Operation 
(16mm 
Projector) 



ES ES % Cost 
ConrtSP CtrlSD Imor. Rario R r f rrP ^ 



Equipment 
Knowledge 



IVDvs 
Classroom 

IVDvs 
On- Job Tmg 

Classroom vs 
On-Job Tmg 



Tutorial 105Exp IVDvs 
104Ctri Videotape 



Tutorial 30Expl Branched IVD 
30Exp2 (Expl)vs 
30Ctrll Linear IVD 
30Ctrl2 (Exp2)vs 

Branched Video- 
tape(Ctrll)v. 
Linear Videotape 
(Ctrl2) 



Branched IVD vs -.28 TJ 
Linear IVD 

Branched IVD vs .85™ 
Branched Videotape 



.76 CK 
p<.05 

1.53 CK 
p<.001 

.82 
p<.05 

1.145" 



Errors on 1.20 r K 
Content Test P<.001 



.75 20 

1.42 58 

.77 31 
1.085 

1.10 47 



Recall of 
Device 
Component 
Names 



ANOVA 
(Delivery 
System 
p<.001) 

(Branching 
ns) 



-.28 
ns 

.70 
p<.01 



na Bosco and 

Wagner (1988) 

na Fowler (1980) 



15 



Notes Concerning Effect Size: 

T - Tutorial Approach; C ~ Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure- 
P - Performance Measure; R -- Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX « Peformante ' 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I - Interactivity Comparison- DC - 
Interactivity Average; B ~ Baseline Comparison; BX - Baseline Average 
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Table A- 1 (Continued). Studies of Level m Videodisc Instruction. 

Instructional 



Instructional Instructional 
Setting Content 



Approach H 



Comparison Outcome 



Branched IVDvs .27 T K 
Linear Videotape 

Linear IVDvs 
Branched Videotape 

Linear IVDvs 
Linear Videotape 

Branched Videotape 
vs Linear Videotape 



Instruction 

Completion 

Time 



Branched IVD 
vs Linear IVD 

Branched IVDvs 1.68 TD 
Branched Videotape 

Branched IVDvs .97 TD 
Linear Videotape 

Linear IVD vs 
Branched Videotape 



ES ES % Cost 
CombSD CtrlSD Impr. Ratio Reference 



.21 
ns 

1.07 
p<.01 

.46 
ns 

-.41 
ns 

ANOVA 
(Delivery 
System 

JK.001) 

(Branching 
ns) 

-.04™ 
ns 



.88 19 



.36 8 



-.38 -9 



-.05 -1 



1.51 
p<.01 

.84 
p<.01 

1.77 
p<.01 



27 



18 



1.54 27 



Notes Concerning Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure; 
P - Performance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX - Peformance 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I - Interactivity Comparison; IX - 
Interactivity Average; B - Baseline Comparison'; BX - Baseline Average 
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Table A-l (Continued). Studtet of Level HI Videodisc Instruction. 

Companion 



Instructional Instructional 

Sstdni Qddjbu 



Instructional 
Apprmch tL 



ES ES % Cost 
_Qu£2aD£ CombSD CtriSD Imor. Ratio Reference 



96 



9 

ERIC 



Linear IVDvs 
Linear Videotape 

Blanched Videotape 
vs Linear Videotape 



28Expl Opearnon, 

27Exp2 Fault 

24Ctrll Location, 

23Ctrl2 and Transfer 



Branched IVDvs 
Linear IVD 



ANOVA 
(Delivery 
System 
p<.05) 

(Branching 
p<.01) 

.72™ 



Branched IVD vs .S9 T P 
Branched Videotape 

Branched IVDvs 1.02 TP 
Linear Videotape 



1.04 
p<.01 

-.59 
p<.05 

.56™ 

1.325 DX 
<94 ax 



.88 

-.57 

.455 
1.175 
.815 



19 



-12 



.16 K -.12 



.75 
p<.05 

.58 
p<.05 

.99 
p<.01 



16 



13 



26 



0 7 



Notes Concerning Effect Size. 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K •- Knowledge Measure; 
P - Performance Measure; R - Retention; D - Time to Complete Instruction; KX Knowledge Average; PX - Peformance 
Avenge; RX - Retention Average; AX - Overall Avenge (for K, P, R, A D Measures); I - Interactivity Comparison; IX - 
Interactivity Average; B - Baseline Comparison; BX - Baseline Average -- • - 



Table A - 1 (Continued). Studies of Lev el m Videodisc Instruction. 

Comparison Ojh&c»% 



Instructional 
SfitHDC 



Instructional 
Co"* 6 "* 



Instructional 
Approach N 



ES ES % Cost 
Comb SD Ctrl SD Impr. Ratio Reference 



Higher Biology Tutorial 
Education ^development 

of Living 

Things" 



25Exp 
24Ctrl 



Linear IVDvs 
Branched Videotape 

Linear IVDvs 
Linear Videotape 

Branched Videotape 
vs Linear Videotape 



IVDvs 
Lecture 
(Student 
Volunteers) 



Biology 
Knowledge 
(Objective 
Test) 



-.10 
ns 

.36 
ns 

.43 
ns 

.805™ 

.30™ 
ns 



Biology .88™ 
Knowledge p<.05 
(Short Answer 
Test) 



lWk 
Retention 
(Objective 
Test) 



.27™ 
ns 



1 Wk .49™ 
Retention ns 
(Short Answer 
Test) 



-.09 -2 

.34 9 

.43 11 
.785 

.27 8 



.93 73 



na Bunderson, 
Baillio, Olsen, 
Lipson, and 
Fisher (1984) 

Also in 

Bunderson, 
Olsen, and 
Baillio (1981) 



.29 6 



.53 36 




Notes Concerning Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure; 
P -- Performance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX - Peformance 
Average; RX - Retention Average; AX - Overall Average (for K, P. R, & D Measures); I -- Interactivity Comparison; IX - 
Interactivity Average; B - Baseline Comparison; BX - Baseline Average 
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Table A- 1 (Continued). Studies of Level III Videodisc Ii 



!iM'!»>i!Wl' 



Instructional Instnictional 
SiStiOg Content 



Instructional 
Approach fcL 



Comparison Outcome 



ES ES % Cost 
Comb SD Ctrl SD Impr. Ratio Reference 



28Exp IVDvs 
25Ctrl Lecture 
(Utah) 
(Random 
Assignment) 



Learning .78 T>D 
Time ns 

.59** 

.38 RX 

.54^ 

Biology .47 TK 
Knowledge p<.05 
(Objective 
Test) 

Biology .64 T,K 
Knowledge p<.05 
(Short Answer 
Test) 

Learning 
Tune 



2.04 T - D 
p<.05 

.555^ 

1.05** 



.60 32 

.60 
.41 
.52 

.88 18 



.58 37 



2.12 44 

.73 
1.19 



na 



101 

Notes Concerning Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure; 
P Performance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX - Pcformance 
Average; RX - Retention Average; AX - Overall Average (tor K, P, R, & D Measures); I - Interactivity Comparison; IX - 
Interactivity Average; B - Baseline Comparison; BX ~ Baseline Average 
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Table A-l (Continued). Studies of Level HI Videodisc Instruction. 



Instructional Instructional Instructional 
Setting Content ApjBBKh fcL 



Comparison OuKPmC 



ES ES % Cost 
Comb SD Ctrl SD Imor. Ratio Reference 



24Exp 
73Ctri 



IVDvs 

Lecture 

(Texas) 

(Random 

Assignment) 



Higher 
Education 



Foreign 
Language 
(Beginning 
French) 



Biology .81™ 

Knowledge p<.05 
(Objective 
Test) 

Biology .58™ 

Knowledge p<.05 
(Short Answer 
Test) 

Learning 1.10 TD 

Thne p<.05 

.83^ 

Tutorial 26Exp IVD(Expl)vs Course (ANOVA) 
26Ctril aassroora(Ctrll) Knowledge p<.001 
26Ctrl2 vs No Instruction (Multiple 

(Ctrl2) Choice Items) 



IVDvs 
Classroom 

IVDvs 

No Instruction 

Classroom vs 
No Instruction 



.08™ 
ns 

1.23™ 
p<.05 

1.31 
p<.05 



.45 



.09 
1.50 
1.40 



6 na 



.59 17 



1.13 32 

.52 
.72 



2 
31 
29 



na Crotty (1984) 



Notes Concerning Effect Size: , , , 

T - Tutorial Approach; C Tutorial and Simulation Approach Combined; S -- Simulation Approach; K -- Knowledge Measure; 
P - Performance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX -- Peforrnance 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I - Interactivity Comparison; DC -- 
cd?/- Interactivity Average; B - Baseline Comparison; BX~ Baseline Average - - 
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Table A- 1 (Continued). Studies of Level m Videodisc Instruction. 



Instructional Instructional 
Settiny Content 



Instructional 
Approach N 



Higher Physics Simulated 21Exp 

Education "Puzzle of die Laboratory 16Ctrl 
Tacoma Bridge 

Collapse" 22Exp 

19Ctrl 

Higher Biology Simulated 22Exp 

Education (Respiration) Laboratory 8Ctrl 



Comparison Outcome 



ES ES % Cost 
CombSD CtrlSD Impr. Ratio Reference 



IVDvs 
Classroom vs 
No Instruction 



IVDvs 
Classroom 

IVDvs 

No Instruction 

Qassroornvs 
No Instruction 



IVDvs 

Laboratory 

Equipment 



IVDLabvs 

Traditional 

Lab 



Course 
Knowledge 
(Completion 
Items) 



Physics 
Knowledge 

Scientific 
Attitudes 

Conteii 
Test (Paper) 



(ANOVA) 
p<001 



1.16™ 1.02 64 
p<.05 

2.58 T - B 2.52 294 
p<05 



1.03 
p<.05 



29 s.K 
ns 

.26 
ns 



1.20 139 



.62 KX .555 
1.905 BX 2.01 



.29 
.24 



11 nu Stevens (1984) 
2 



-.29 SK -.34 -8 
ns 



.80 Davis (1985) 
CD 

.02 
(OS) 



Biology 



22Exp Same 



Same 



IMP* 



.83 23 na 
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Notes Concerning Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K -- Knowledge Measure; 
P -- Performance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX - Peformance 
Avenge; RX ~ Retention Average; AX - Overall Average (for K, P, R, & D Measures); 1 - Interactivity Comparison; DC - 
Interactivity Average; B - Baseline Comparison; BX -- Baseline Average 



Table A-l (Continued). Studies of Level HI Videodisc Instruction. 

Comparison 



Instructional Instructional 

Sating Content 



Higher 
Education 



Higher 
Education 



Higher 
Education 



Instructional 
.Aiamacli CL 



ES ES % 
Outcome CombSD CtrlSD Imnr. 



Cost 

Ratio Reference 



(Climax and 
Life) 

Chemistry 
(Kinetics and 
Equilibrium, 



25Ctrl 



Tutorial with 2^Exp 
Simulated 23Ctrl 
Laboratory 



Public Health 
(CPR 

Instruction) 



Simulation 48 Exp 
and Sensored SICtrl 
Manikin 



p<.01 



MHLab Chemistry .69 c * na 

vs Lab only Knowledge p<.05 
(Lab Reports) 

Course (ANOVA) na 
Knowledge pc.OOl 



21Expl IVD(Expl)vs 
17Exp2 IVD+Lab (Exp2) 
49Ctrl vs Lab only (Ctrl) 

IVDvs 
Lab only 

IVD+Lab 
vs Lab only 

IVD+Lab 
vsIVD 



Chemistry Tutorial with 25 Exp IVDvs 
(Kinetics and Simulated 27Ctri Lab 
Equilibrium) Laboratory 



IVDvs 

Classroom 

Instruction 



Course 
Knowledge 



Course 
Knowledge 



1.16** 
p<05 

LOOP* 
p<.01 

.19 
ns 

1.0%** 

.65 c * 
p<.05 



.35 s * 
ns 



na 



17 



1.08 40 

.94 34 

.19 4 
1.01 

.66 22 



na Smith, Jones, 
and Waugh 
(1986) 



na 



na Jones (1987) 
Also in 
Smith and 
Jones (1989) 

na Lyness (1987) 



Performance .39 s,p na 



47 



ERIC 



Notes Concerning Effect Size: ^ . , J m- 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K -- Knowledge Measure; 
P Performance Measure; R - Retention; D - Time to Complete Instruction; KX -- Knowledge Average; PX - Peformance 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I - Interactivity Comparison; DC » 
Interactivity Average; B - Baseline Comparison; BX -- Baseline Average ..... ^_ ^ 



Tible A-l (Continued), StmBct of Level m VWHim InTtnifftiffll 

Instructional 



tattructtonal Instnictiontl 
SfittiOf Content 



Approtch fcL 



108 



Outcome 


ES ES % 
Comb SD CtrlSD Imor. 


(Single 
Rescuer) 


ns 






Performance .23 s,p 
(TWo Rescuer) ns 


na 


19 


Performance 
(Obstructed 
Airway #1) 


.41 s - p 
p<05 


na 


23 


Pfcrfonutnce 
(Obstructed 
Airway #2) 


.80 SP 
p<.05 


na 


63 


Pufmiuincc 
(Obstructed 
Airway #3) 


.67 sp 
p<05 


na 


59 


Performance 
(Infant) 


-.27 s - p 
ns 


na 


-23 


Performance 
(Infant 
Obstructed 
Airway #1) 


-.31* p 
ns 


na 


-12 


Performance 
(Infant 
Obstructed 
Airway #2) 


.08 s - p 
ns 


na 


6 



Cost 



.25 PX 



na 



ioy 




Notes Ctacermng Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure; 
P - Pafumance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX - Peformance 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I Interactivity Comparison; IX - 

Interactivity Average; B-^Baseline Comparison; BX - Baseline Average 
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Table A- 1 (Continued). Studies of Level Til Videodisc Instruction . 

Comparison 



Instructional Instructional 
SfitiDg Cjonjenj 



Instructional 



ES ES % Cost 
Outcome Comb SD Ctrl SD Imnr. Ratio Reference 



Higher 
Ed ucati o n 



Foreign Tutorial 2 jcpl IVD Linear Course 

Language UExp2 (Expl)vsIVD Knowledge 

(Beginning 23Exp3 Segmented 

Spanish) 23Ctrl (Exp2)vsIVD 



.26** 

(ANOVA) 
p<.001 



na 



Interactive (Exp3) 
vs Irrelevant Ins- 
truction(Cbi) 










IVD Interactive 
vs IVD Segmented 




1.36™ 
p<.001 


1.41 


33 


IVD Interactive 
vs IVD Linear 




2.16 TJ 
p<001 


2.49 


58 


IVD Segmented 
vs IVD Linear 




.72™ 
ns 


.80 


18 


IVD Interactive 
vs Irrelevant Inst 


p<01 


2.94 TB 


3.84 


88 


IVD Segmented 
vs Irrelevant Inst. 


p<.05 


1.44 TB 


1.80 


41 


IVD Linear vs 
Irrelevant Inst 




.75 
ns 


.83 


19 


IVD Interactive 
vsIVD 


Course 
Knowledge 


.74™ 
p<.001 


.79 


30 



na Verano 
(1987) 



9 

ERIC 
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Notes Concerning Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K Knowledge Measure; 
P - Performance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX - Peformance 
Avenge; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I - Interactivity Comparison; DC - 
Interactivity Average; B - Baseline Comparison; BX - Baseline Average 
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Table A-l (Continued). Studies of Level m Videodisc ln«tn.rrinn 



112 



Instructional 
Setting 



Instructional 
Content 



Instructional 
Approach N 



Higher Chemistry Tutorial with 26Exp 
Education (Gas Analysis) Simulated 22Cbi 

Laboratory 



Comparison 



Segmented 



Outcome 
(4 Wk Delay) 



ES ES % Cost 
CombSD CtrlSD Imor. Ratio Reference 



IVD Interactive 1.41 TJ 2.24 56 

vsIVD Linear jk.001 

IVD Segmented .S6 TJ .82 21 

vsIVD Linear ns 

IVD Interactive 1.33 T « B 2.06 51 

vs Irrelevant Inst p<.001 

IVD Segmented .47™ .68 17 

vs Irrelevant Inst ns 



IVD Linear vs 
Irrelevant Inst 



IVDvs 
Lab 



-.13 
ns 

1.16" 

1.545 BX 

Spectrometer 1.50°'* 
Usage -Score p<.01 
(3 Wk Delay) 

Spectrometer 1.37 C,R 

Usage -Time p<.01 
to Complete 
(3 Wk Delay) 



-.12 -3 

1.425 
2.095 

1.08 69 

1.09 39 



na Jones (1988) 



J J 3 



Notes Concerning Effect Size: 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K -- Knowledge Measure; 
P - Performance Measure; R - Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX - Peformance 
Average; RX - Retention Average; AX - Overall Average (for K, P, R, & D Measures); I - Interactivity Comparison; IX - 
Interactivity Average; B - Baseline Comparison; BX - Baseline Average 
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Table A-l (Continued). Studies of Level m Videodisc Instruction 



Instructional Instructional 
StitiDg CjQDJ&ni 



Instructional 
Approach N_ 



Comparison Outcome 



ES ES % Cost 
Comb SD Ct rl SD Imor. Ratio Reference 



Higher Medical 
Education Education 
(Trauma) 



Simulated 
Patient with 
Critique of 
Student 
Performance 



Knowledge 
Test(2Wk 
Delay) 



33 Exp Standard Knowledge 
31 Ctrl Instruction with Test (Paper) 

vs without IVD 

(1 rVD Program) 



28 Exp Standard Knowledge 
40 Ctrl Instruction with Test (Paper) 

vs without IVD 

(4 IVD Programs) 



.50 CR 
p<01 

1.12 RX 

1.01 SB 
p<.01 



1.09 s8 
p<.01 



.51 10 



.89 

.97 16 



na Allan (1989) 



1.12 20 



Notes Concerning Effect Size: L „ ^ . . w 

T - Tutorial Approach; C - Tutorial and Simulation Approach Combined; S - Simulation Approach; K - Knowledge Measure; 
P - Performance Measure; R -- Retention; D - Time to Complete Instruction; KX - Knowledge Average; PX - Peformance 
Average; RX ~ Retention Average; AX - Overall Average (for K, P, R, & D Measures); I -- Interactivity Comparison; IX - 
P R?r Interactivity Average; B - Baseline Comparison; BX -- Baseline Average 
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APPENDIX B 
STUDIES OF LEVEL II VIDEODISC INSTRUCTION 
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Table B-l. Studies of Level n Videodisc Instruction. 
Instroctional Instructional Instructional 



ES 



ES 



Cost 



Sefiinf 


Content 


Approach 


N 


Comparison 


Outcome 


Comb SD Ctrl SD 


Impr. 


Ratio Reference 




Armor 


Slide/Tape 


16Exp 


IVDvs 


Performance 


-2.42 p 


-2.69 


-17 


na Holmgren, 


Training 


Concepts 


Tutorial 


31Ctrl 


Slide/Tape 


Test 


p<.05 






Dyer, 








.69? 






Hilligoss, and 




A .'M 

Artillery 




8Exp 






.85 


o 
8 


Heller (ly/v) 




Concepts 




33Ctrl 






ns 










Infantry 




17Exp 










-18 






Concepts 




31Cirl 






p<.05 










Armor 




16Exp 


IVD with Re- 


Performance 


—2.83 


i no 
-3.08 


-20 






Concepts 




31 Ctrl 


view vs 


Test 


p<.05 














Slide/Tape 














A - *** 

Artillery 




23Exp 






-65 p 


-.69 


—l 






Concepts 




33Cui 






ns 










Infantrv 




15Exp 






-1.93 p 


-2.20 


-15 






Concepts 




31Ctri 






p<.05 








MiUtaiy 


Electronics 


Slide/Tape 


72Exp 


IVDvs 


Paper Test of 


-.07 K 


na 


-1 


na King (1982) 


Training 


Maintenance 


Tutorial 


163Ctri 


Slide/Tape 


Knowledge 


ns 










with 








.29° 












Actual 


72Exn 




UVWvl 1 




9 








ojuipnicni 


i03i*in 






p<.05 
















Time 1 
















70Exp 




Lesson 


-.36° 


-.40 


-11 










158Ctrl 




Completion 


p<.05 


















Time 2 











Notes Concerning Effect Size: 

K - Knowledge Measure; P - Performance Measure; D - Time to Complete Instruction 
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Tabic B-l (Continued). Studies of Level H Videodisc Instnicrinn 

Comparison 



Instructional Instructional 
Sating Content 



Instructional 
Approach N 



^ ES ES % Cost 

-QufcOm CombSD CItISD Imnr. Rarin Rrf^ 



69Exp 
157Ctri 



Lesson 
Completion 
Tune 3 



-.13 d -.17 



ns 



-7 



Higher Physics Simulated 18Exp IVDvs 

Education "Puzzle of the Laboratory 18Ctrl Laboratory 

Tacoma Bridge Equipment 

Collapse" 



Physics 
Knowledge 
(Standing 
Waves) 



-.17 K 
ns 



na 



na Wankel (1984) 



Higher 
Education 



Chemistry 



Tutorial 58Exp 
with 54Ctrl 
Simulated 
Laboratory 

57Exp 
55Ctrl 



IVDvs 
Videotape 



Course 
Knowledge 



Graphical 
Analysis 



.39 K 
p<.05 

.31* 
ns 



.47 



.30 



56 na 



10 



Russell, 
Staskun, and 
Mitchell 
(1985) 



119 



tin 



9 

ERIC 



Notes Concerning Effect Size: 

K -- Knowledge Measure; P -- Performance Measure; D Time to Complete Instruction 
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